Can LLMs Replace Economic Choice Prediction Labs? The Case of Language-based Persuasion Games

ACL ARR 2024 August Submission140 Authors

Abstract:

Human choice prediction in economic contexts is crucial for applications in marketing, finance, public policy, and more. This task, however, is often constrained by the difficulties in acquiring human choice data.
With most experimental economics studies focusing on simple choice settings, the AI community has explored whether LLMs can substitute for humans in these predictions and examined more complex experimental economics settings. However, a key question remains: can LLMs generate training data for human choice prediction? We explore this in language-based persuasion games, a complex economic setting involving natural language in strategic interactions. Our experiments show that models trained on LLM-generated data can effectively predict human behavior in these games and even outperform models trained on actual human data.

Paper Type: Long

Research Area: Human-Centered NLP

Research Area Keywords: Human-Centered NLP, human-AI interaction, value-centered design, human-centered evaluation

Contribution Types: Model analysis & interpretability, Approaches to low-resource settings, Data resources, Data analysis

Languages Studied: English

Submission Number: 140