Estimating the Impact of Coordinated Inauthentic Behavior on Content Recommendations in Social Networks

Swapneel S Mehta, Atilim Gunes Baydin, Bogdan State, Richard Bonneau, Jonathan Nagler, Philip Torr

Keywords: coordinated inauthentic behavior, agent based modeling, recommender systems, social networks

TL;DR: Simulating the effects of coordinated activity promoting malicious and misleading content amplified through recommendation systems on social media

Abstract:

Online disinformation is a dynamic and pervasive problem on social networks as evidenced by a spate of public disasters in light of active efforts to combat it.
Since the massive amounts of content generated each day on these platforms is impossible to manually curate, ranking and recommendation algorithms are a key apparatus that drive user interactions.
However, the vulnerability of ranking and recommendation algorithms to attack from coordinated campaigns spreading misleading information has been established both theoretically and anecdotally.
Unfortunately it is unclear how effective countermeasures to disinformation are in practice due to the limited view we have into the operation of such platforms.
We develop a multiagent simulation of a popular social network, Reddit, that aligns with the state-action space available to real users based on the platform's affordances.
We collect millions of real-world interactions from Reddit to estimate the network for each user in our dataset and utilise Reddit's self-described content ranking strategies to compare the impact of coordinated activity on content spread by each algorithm.
We expect that this will inform the design of robust content distribution systems that are resilient against targeted attacks by groups of malicious actors.