Who Gets Which Message? Auditing Demographic Bias in LLM-Generated Targeted Text

Tunazzina Islam

Keywords: Large language models (LLMs), microtargeting, demographic bias, gender bias, age bias, persuasion bias, bias audit, climate communication

TL;DR: Large language models adapt persuasive framing based on demographic conditioning, with contextual prompts amplifying age- and gender-based differences

Abstract: Large language models (LLMs) are increasingly capable of generating personalized, persuasive text at scale, raising new questions about bias and fairness in automated communication. This paper presents the first systematic analysis of how LLMs behave when tasked with demographic-conditioned targeted messaging. We introduce a controlled evaluation framework using three leading models—GPT-4o, Llama-3.3, and Mistral-Large 2.1—across two generation settings: **Standalone Generation**, which isolates intrinsic demographic effects, and **Context-Rich Generation**, which incorporates thematic and regional context to emulate realistic targeting. We evaluate generated messages along three dimensions: *lexical content*, *language style*, and *persuasive framing*. We instantiate this framework on ***climate communication*** and find consistent age- and gender-based asymmetries across models: male- and youth-targeted messages emphasize *agency*, *innovation*, and *assertiveness*, while female- and senior-targeted messages stress *warmth*, *care*, and *tradition*. Contextual prompts systematically amplify these disparities, with persuasion scores significantly higher for messages tailored to younger or male audiences. Our findings demonstrate how demographic stereotypes can surface and intensify in LLM-generated targeted communication, underscoring the need for bias-aware generation pipelines and transparent auditing frameworks that explicitly account for demographic conditioning in socially sensitive applications.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 11