Abstract
The Internet has given new opportunities to those who wish to interfere and disrupt society through the systematic manipulation of social media. One goal of these cyber-enabled information operations is to increase polarisation in Western societies by stoking both sides of controversial debates. Whether these operations are successful remains unclear. This paper describes how novel applications of computational techniques can be used to test the impact of historical activity from the Russian Internet Research Agency (IRA) on two social media platforms: Twitter and Reddit. We show that activity originating from the Russian IRA had a measurable effect on the subsequent conversations of genuine users. On Twitter, increases in Russian IRA activity predicted subsequent increases in the degree of polarisation of the conversation surrounding the Black Lives Matter movement. On Reddit, comment threads started by Russian IRA accounts contained more toxic language and identity-based attacks. We use causal analysis modelling to further show that Russian IRA activity in existing threads caused measurable changes in the conversational quality of the following 25-100 posts. By developing methods to measure the impact of information operations in online conversations and demonstrating a measurable effect on genuine conversations, our study provides an important step in developing effective countermeasures.
Keywords—information operations, social media, social psychology, group polarisation, disinformation, strategic communications
About the Authors
John D. Gallacher is a Cyber Security DPhil student at the University of Oxford working with the department of Experimental Psychology and the Oxford Internet Institute.
Dr Marc W. Heerdink is an Assistant Professor of Social Psychology at the University of Amsterdam, where he also obtained his PhD, and a former visiting postdoc at the University of Oxford.
Bibliography
Althoff, Tim, Pranav Jindal, and Jure Leskovec. ‘Online Actions with Offline Impact: How Online Social Networks Influence Online and Offline User Behaviour’, WSDM Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, 2017, 537–46.
Arif, Ahmer, Leo G Stewart, and Kate Starbird. Acting the Part: Examining Information Operations within #BlackLivesMatter Discourse’, Proceedings of the ACM on Human-Computer Interaction Vol. 2, Issue CSCW, Article № 20 (2018): 1–26.
Arnaudo, Dan. ‘Computational Propaganda in Brazil: Social Bots during Elections’, University of Oxford Computational Propaganda Research Project 8 (2017): 1–39.
Bail, Christopher, Lisa Argyle, Taylor Brown, John Bumpus, Haohan Chen, M.B. Hunzaker, Jaemin Lee, Marcus Mann, Friedolin Merhout, and Alexander Volfovsky. ‘Exposure to Opposing Views Can Increase Political Polarization: Evidence from a Large-Scale Field Experiment on Social Media’, Proceedings of the National Academy of Sciences, 2018, 1–6.
Bakshy, E., S. Messing, and L. A. Adamic. ‘Exposure to Ideologically Diverse News and Opinion on Facebook’, Science Vol. 348, Issue 6239 (2015): 1130–32.
Barberá, Pablo. ‘Explaining the Spread of Misinformation on Social Media: Evidence from the 2016 U.S. Presidential Election’, Comparative Politics Newsletter Vol. 28, Issue 2 (2018): 7–11.
Barberá, Pablo, John T Jost, Jonathan Nagler, Joshua A. Tucker, and Richard Bonneau. Tweeting From Left to Right: Is Online Political Communication More than an Echo Chamber?’, Psychological Science Vol. 26, № 10 (2015): 1531–42.
Barberá, Pablo, and Gonzalo Rivero. ‘Understanding the Political Representativeness of Twitter Users’, Social Science Computer Review Vol. 33, № 6 (2015), 712–29.
Bay, Sebastian, Giorgio Bertolin, Nora Biteniece, Edward H Christie, E Rolf, John D Gallacher, Kateryna Kononova, and Tetiana Marchenko. Responding to Cognitive Security Challenges. Edited by Anna Reynolds and Mike Collier. Riga, Latvia: NATO Strategic Communications Centre of Excellence.
Berger, J. M., and Jonathon Morgan. ‘The ISIS Twitter Census: Defining and Describing the Population of ISIS Supporters on Twitter’, The Brookings Project on U.S. Relations with the Islamic World, 5 March 2015.
Berinsky, Adam J. ‘Rumors and Health Care Reform: Experiments in Political Misinformation’, British Journal of Political Science Vol. 47, Issue 2 (2017): 241–62.
Bhattacharya, Parantapa, and Niloy Ganguly. ‘Characterizing Deleted Tweets and Their Authors’, Icwsm, 2016, 10–13.
Bishara, Anthony J., and James B. Hittner. ‘Testing the Significance of a Correlation with Nonnormal Data: Comparison of Pearson, Spearman, Transformation, and Resampling Approaches’, Psychological Methods Vol. 17, № 3 (2012): 399–417.
Box, G. E. P., and D. R. Cox. ‘An Analysis of Transformations’, Journal of the Royal Statistical Society Vol. 26, № 2 (1964): 211–43.
Brady, William J., Julian A. Wills, John T. Jost, Joshua A. Tucker, and Jay J. Van Bavel. ‘Emotion Shapes the Diffusion of Moralized Content in Social Networks’, Proceedings of the National Academy of Sciences Vol. 114, № 28 (2017): 7313–18.
Brian, Ripley, Bill Venables, Douglas M Bates, David Firth, and Maintainer Brian Ripley. ‘Package “MASS”’ CRAN (2018).
Bright, Jonathan. ‘Explaining the Emergence of Echo Chambers on Social Media: The Role of Ideology and Extremism’, SSRN Electronic Journal (2016).
Brodersen, Kay H, Fabian Gallusser, Jim Koehler, Nicolas Remy, and Steven L Scott. ‘Inferring Causal Impact Using Bayesian Structural Time-Series Models’, The Annals of Applied Statistics Vol. 9, № 1 (2015): 247–74.
Brodersen, Kay H, and Alain Hauser. ‘Package “CausalImpact”’, CRAN, 2017, 1–8.
Brown, Rupert, and Miles Hewstone. ‘An Integrative Theory of Intergroup Contact’, Advances in Experimental Social Psychology Vol. 37 (2005): 255–343.
Brundage, Miles, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, et al. ‘The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation’, February 2018.
Castree, Noel, Rob. Kitchen, and Alisdair. Rogers. 2013. A Dictionary of Human Geography. Oxford University Press.
Conover, M, J Ratkiewicz, and M Francisco. ‘Political Polarization on Twitter’, Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media, Conference Paper (2011).
Crockett, M. J. ‘Moral Outrage in the Digital Age’, Nature Human Behaviour Vol. 1 (2017):769–71.
Depue, Jacob B., Brian G. Southwell, Anne E. Betzner, and Barbara M. Walsh. ‘Encoded Exposure to Tobacco Use in Social Media Predicts Subsequent Smoking Behavior’, American Journal of Health Promotion Vol. 29, № 4 (2015).
DiResta, Renee, Kris Shaffer, Becky Ruppel, David Sullivan, Robert Matney, Ryan Fox, Jonathan Albright, and Ben Johnson. The Tactics & Tropes of the Internet Research Agency’, New Knowledge Disinformation Report Whitepaper, 2018.
Druckman, James N., Erik Peterson, and Rune Slothuus. ‘How Elite Partisan Polarization Affects Public Opinion Formation’, American Political Science Review Vol. 107, Issue 01 (2013).
Ferrara, Emilio. ‘Manipulation and Abuse on Social Media’, SIGWEB Newsletter, Spring 2015.
Ferrara, Emilio, Disinformation and social bot operations in the run up to the 2017 French presidential election’, First Monday, Vol. 22, № 8, 2017.
Fredheim, Rolf. 2019. ‘Robotrolling 2019, Issue 1’ (Riga, Latvia, NATO StratCom COE, 2019).
Freelon, Deen, Charlton D. McIlwain, and Meredith Clark. ‘Beyond the Hashtags: #Ferguson, #Blacklivesmatter, and the Online Struggle for Offline Justice’, SSRN Electronic Journal, (2016).
Friggeri, Adrien, Lada Adamic, Dean Eckles, and Justin Cheng. ‘Rumor Cascades’, ICWSM 2014 International Conference on Weblogs and Social Media, 2014, 101–10.
Gallacher, John D, and Rolf E Fredheim. ‘Division Abroad, Cohesion at Home: How the Russian Troll Factory Works to Divide Societies Overseas but Spread pro-Regime Messages at Home’, in Responding to Cognitive Security Challenges (Riga, Latvia: NATO StratCom CoE, 2019), 60–79.
Gallacher, John David, Marc W Heerdink, and Miles Hewstone. “Online Contact between Opposing Political Protest Groups via Social Media Predicts Physical Violence of Offline Encounters.” Under Review, 1–44.
Gallagher, Ryan J., Andrew J. Reagan, Christopher M. Danforth, and Peter Sheridan Dodds. ‘Divergent Discourse between Protests and Counter-Protests: #BlackLivesMatter and #AllLivesMatter’, PLoS ONE 13, № 4 (2018): 1–23.
Garimella, Kiran, Gianmarco De Francisci Morales, Aristides Gionis, and Michael Mathioudakis. 2018. ‘Political Discourse on Social Media: Echo Chambers, Gatekeepers, and the Price of Bipartisanship’, International World Wide Web Conference 2 (2018).
Garrett, R. Kelly, Brian E. Weeks, and Rachel L. Neo. ‘Driving a Wedge Between Evidence and Beliefs: How Online Ideological News Exposure Promotes Political Misperceptions’, Journal of Computer-Mediated Communication Vol. 21, Issue 5 (2016): 331–48.
Gerber, Theodore P., and Jane Zavisca. ‘Does Russian Propaganda Work?’, The Washington Quarterly Vol. 39, Issue 2 (2016).
Gideon, Lucian, Conway Iii, Kathrene R Conway, and Shannon C Houck. ‘Automated Integrative Complexity’, Political Psychology Vol. 35, № 5 (2014).
Gleicher, Nathaniel. ‘Removing Coordinated Inauthentic Behavior from the UK and Romania’, Facebook Newsroom, 7 March 2019.
Google Project Jigsaw, ‘Perspective’, accessed 23 March 2018.
Guess, Andrew, Pablo Barber, Cristian Vaccari, United Kingdom, Brendan Nyhan, Alexandra Seigel, Sergey Sanovich, and Denis Stukal. ‘Social Media, Political Polarization, and Political Disinformation: A Review of the Scientific Literature’, The William and Flora Hewlett Foundation, 2018.
Guttieri, Karen, Michael D. Wallace, and Peter Suedfeld. The Integrative Complexity of American Decision Makers in The Cuban Missile Crisis’, Journal of Conflict Resolution Vol. 39, № 4 (1 December 1995).
Gvirsman, Shira Dvir, R. Kelly Garrett, Aysenur Dal, Rachel Neo, Yariv Tsfati, and Benjamin K. Johnson. ‘Implications of Pro- and Counterattitudinal Information Exposure for Affective Polarization’, Human Communication Research Vol. 40, Issue 3 (2014).
Hindman, Matthew, and Vlad Barash. ‘Disinformation, “Fake News” and Influence Campaigns on Twitter’, 2018.
Hirschfeld, H. O., and J. Wishart. ‘A Connection between Correlation and Contingency’, Mathematical Proceedings of the Cambridge Philosophical Society Vol. 31, Issue 4 (October 1935): 520.
Houck, Shannon C., Meredith A. Repke, and Lucian Gideon Conway. Understanding What Makes Terrorist Groups’ Propaganda Effective: An Integrative Complexity Analysis of ISIL and Al Qaeda’, Journal of Policing, Intelligence and Counter Terrorism Vol. 12, Issue 2 (2017): 105–18.
Houck, Shannon C. 2014. ‘Automated Integrative Complexity : Current Challenges and Future Directions’, Political Psychology 35, Issue 5 (2014): 603– 24.
Howard, Philip N, Bharath Ganesh, Dimitra Liotsiu, John Kelly, and Graphika Camille François. ‘The IRA, Social Media and Political Polarization in the United States, 2012-2018’, University of Oxford Computational Research Project, 2018.
Howell, Lee, Martina N Gmur, Peter Bisanz, Isabel de Sola, Karine Von, Benjamin Prampart, Rigas Hadzilacos, Stefan Hall, and Aude Lanois. ‘Outlook on the Global Agenda 2014’ (Geneva: World Econoomic Forum, 2014).
Husson, Francois, Julie Josse, Sebastien Le, and Jeremy Mazet. ‘Package “FactoMineR”’, CRAN, 2018.
Intelligence Community Assesment. ‘Assessing Russian Activities and Intentions in Recent US Elections’, Office of the Director of National Intelligence, 2017.
Karan, Kanisk, Donara Barojan, Melissa Hall, and Graham Brookie. “#TrollTracker: Outward Influence Operation from Iran.” 2019.
Kello, Lucas. The Virtual Weapon and International Order (Yale University Press, 2017).
King, Gary, Pan Jennifer, and Roberts Margaret E. ‘How the Chinese Government Fabricates Social Media Posts for Strategic Distraction, Not Engaged Argument’, American Political Science Review Vol. 111, Issue 03 (2017): 484–501.
