SurveyPilot: Finite-State Orchestrated Agentic Framework for Automated Human Opinion Collection from Social Media

ACL ARR 2025 February Submission4902 Authors

Abstract: Opinion survey research is a crucial method used by social scientists for understanding societal beliefs and behaviors. Traditional methodologies often entail high costs and limited scalability, while current automated methods such as opinion synthesis exhibit severe biases and lack traceability. In this paper, we introduce SurveyPilot, a novel finite‐state orchestrated agentic framework that automates the collection and analysis of human opinions from social media platforms. SurveyPilot addresses the limitations of pioneering approaches by (i) providing transparency and traceability in each state of opinion collection and (ii) incorporating several techniques for mitigating biases, notably with a novel genetic algorithm for improving result diversity. Our extensive experiments reveal that SurveyPilot achieves a close alignment with authentic survey results across multiple domains, observing average relative improvements of 68,98% and 51,37% when comparing to opinion synthesis and agent-based approaches.

Paper Type: Long

Research Area: Computational Social Science and Cultural Analytics

Research Area Keywords: NLP tools for social analysis, language/cultural bias analysis, quantitative analyses of news and/or social media

Contribution Types: NLP engineering experiment

Languages Studied: English, French, Italian, Spanish, German

Submission Number: 4902