Go to ICLR 2026 Workshop AFAA homepageAds in ChatGPT? An Analysis of How Large Language Models Navigate Conflicts of Interest
Track: Main Papers Track (6 to 9 pages)
Keywords: Large Language Models, Cognitive Science, Communicative Norms, Values, Dilemmas, Fairness, Evaluation, Advertisements
Abstract: LLMs are typically trained to align with user preferences, through methods such as reinforcement learning (e.g., RLHF). Yet models are beginning to be deployed not merely to satisfy users, but to also generate revenue for the companies that created them through advertisements. This creates the potential for LLMs to face conflicts of interest, where the most beneficial response to the user may not be aligned with the company's best interest. For instance, a promoted product may be more expensive but otherwise equal to another; in this case, what does (and should) the LLM recommend to the user? In this paper, we analyze how LLMs handle these conflicts, using advertisements as a case study. We use work from linguistics on the norms of conversation to identify the ways in which different incentives might lead LLMs to change the way they interact with users, and present a suite of evaluations to characterize these tradeoffs. Through these evaluations, we find that a majority of LLMs forsake user welfare for company incentives---partially modulated by users' socio-economic status and inference-time reasoning---though the degree of such modulation lies on a highly polarized spectrum. We observe similarly polarized behaviors in a subsequent experiment probing for propensities of ``upselling'' sponsored products and unilaterally concealing prices. Overall, our work highlights some of the hidden risks to users that can emerge when companies begin to subtly incentivize LLM-powered advertisers.
Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.
Submission Number: 51
Loading