Text2Decision: Decoding Latent Variables in Risky Decision Making from Think Aloud Text
Keywords: Think Aloud; Large language models; Neural Networks; Risky Decision Making
TL;DR: The study introduces Text2Decision, a model using AI to quantitatively analyze think-aloud protocols, successfully decoding risky decision-making tendencies in both humans and GPT-4 simulations, highlighting the potential of AI in cognitive research.
Abstract: Understanding human thoughts can be difficult, as scientists usually rely on observing behaviors. The think-aloud protocol, where people talk about their thoughts while making decisions, provides a more direct way to study thoughts. However, past research on this topic has mostly been qualitative. Recent advancements in artificial intelligence and natural language processing provide the potential for more quantitative analysis of language data. This study introduces Text2Decision, a model trained on task questions from a large-scale task collection, used to decode decision tendencies in risky decision-making from think-aloud texts. We test our model in both human and GPT-4 simulated think-aloud text data about risky decision-making, which are out-of-distributed in the training. Our findings demonstrate the model's performance in capturing GPT-4 manipulated decision personas and in unveiling heuristic decision tendencies from humans. Text2Decision demonstrates its capability by training on basic task outlines and theoretical frameworks and generalizing to unseen empirical think-aloud text data. This not only allows decoding individual differences from these texts but also extends to analyzing large-scale domain datasets. This study shed light on AI integration in cognitive research for the AI4Science paradigm.
Submission Track: Original Research
Submission Number: 174
Loading