Cognitive Distortion Detection with LLM-generated Datasets

ACL ARR 2025 May Submission684 Authors

Abstract: We present a novel framework for simulating and detecting cognitive distortions (CoDs) in therapist–patient dialogues using large language models (LLMs) and structured therapeutic simulations. By creating individualized distortion profiles for patients and prompting LLMs based on Cognitive Behavioral Therapy (CBT) principles, we simulate therapy sessions that undergo iterative refinement in a reward-guided loop, maximizing naturalness, coherence, and alignment with targeted distortions. We then introduce inline CoD annotations as weak supervision and assess their effect on classifier performance. Leveraging both LLM-simulated sessions and a public CoD dataset through hybrid embeddings, our approach achieves a 0.74 weighted F1. These findings highlight the promise of controlled simulation and iterative reinforcement to boost data-scarce clinical NLP tasks.

Paper Type: Long

Research Area: Machine Learning for NLP

Research Area Keywords: reinforcement learning, few-shot learning, generative models, data augmentation, representation learning

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches to low-resource settings, Data resources

Languages Studied: English

Submission Number: 684