TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots

ICLR 2026 Conference Submission16172 Authors

Keywords: alignment, llm, therapy, evaluation, safety

Abstract: Large language models (LLMs) are increasingly used for mental-health support, yet prevailing evaluation methods—fluency metrics, preference tests, and generic dialogue benchmarks—fail to capture the clinically critical dimensions of psychotherapy. We introduce TherapyGym, a framework that evaluates and improves therapy chatbots along two pillars drawn from clinical science: fidelity and safety. Fidelity is operationalized through the Cognitive Therapy Rating Scale (CTRS), adapted into an automatic pipeline that scores both adherence to CBT techniques and therapist competence across multi-turn interactions. Safety is assessed using a multi-label annotation scheme over conversations, covering domain-specific risks(e.g., judgmental behavior, failure to address harm). To mitigate bias and unreliability in LLM-based judges, we further release TherapyJudgeBench, a validation set comprising 116 dialogues and 1,270 expert ratings, enabling systematic auditing and calibration of judge performance against licensed clinicians. Beyond evaluation, TherapyGym functions as a training harness: CTRS- and safety-derived signals serve as rewards in an RL loop where an LLM therapist engages programmable, realistic patient simulations spanning symptom profiles and conversational styles. Empirically, models trained in TherapyGym achieve higher automatic CTRS scores with improvements that transfer to expert human ratings, demonstrating gains in both clinical skill and safety. Our contributions establish a scalable, clinically grounded pathway for developing therapy chatbots that are not merely conversationally fluent but also faithful to evidence-based practice and responsible in high-stakes use.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 16172