Governing Manipulative and Synthetic Content on Social Media Platforms

Smirity Kaushik, Madelyn Rose Sanfilippo

Abstract: Social media platforms are immensely popular among young adults (13-20 years old). However, these platforms also pose distinct risks, including fraudulent advertisements and scams. The rise of Generative AI (GenAI) based advertising has exacerbated these risks, including deepfake scams. Using the integrated Governing Knowledge Commons-Contextual Integrity (GKC-CI) framework, we perform a structured content analysis of the policies from three stakeholders: social media platforms, foundational GenAI model providers, and GenAI-based advertising services. We analyze how these entities define and constrain the use of GenAI in social media advertising, with particular attention to protection for young adults against manipulative and synthetic GenAI content. We found that current governance lacks enforceable rules regarding GenAI, and are especially lacking in protecting young adults. We discuss implications for platforms and regulators to strengthen institutional governance for GenAI-based advertising targeting young adults.