Using Large Language Models to Detect Outcomes of Qualitative Studies on Adolescent Depression

Alison W. Xin, Dylan Nielson, Karolin Krause, Guilherme Fiorini, Nick Midgley, Francisco Pereira, Juan Antonio Lossio-Ventura

Keywords: Large language models, BERT, Llama 2, Llama 3, adolescent depression, depression outcomes, mental health

Abstract: Depression treatment studies often focus exclusively on changes in depressive symptoms, such as low mood, anhedonia, or sleep disruption. However, incorporating other outcomes important to those experiencing depression, such as the quality of interpersonal relationships or quality of life, could improve understanding of the impacts of depression and effectiveness of treatment. Drawing on data from in-depth interviews with adolescents, parents, and therapists, clinicians produced a novel coding framework that covers additional domains of interest that matter to adolescents, such as relationships, functioning, and well-being. In this paper, we examine whether large language model embeddings can be used to classify the outcomes of this framework from annotated interviews. We compare the suitability of four language models across three different segmentations of interview transcripts, such as conversation turns or non-interviewer utterances. The level of performance achieved by our models makes them useful for a variety of applications, ranging from aiding human annotation of text transcripts to quantifying the presence of outcomes for downstream uses, such as estimating treatment effects or building prognostic models.

Submission Number: 29