GPT Models Can Perform Thematic Analysis in Public Health Studies, Akin to Qualitative Researchers

Yuyi Yang, Charles Alba, Chenyu Wang, Xi Wang, Jami Anderson, Ruopeng An

Abstract: Conducting thematic analysis in qualitative research can be laborious and time-consuming. We propose and evaluate the feasibility of using Generative Pre-trained Transformer (GPT) models to assist public health researchers in extracting themes from interview transcripts. Carefully engineered prompts were used to sequentially extract and synthesize transcripts into a concise set of study-level themes relevant to the study's goals. An evaluation using a 5-point Likert scale (0–4) assessed GPT-generated themes across I I published studies based on four criteria: succinctness, alignment with researcher-identified themes, quality of explanations, and relevance of quotes. Across all four criteria, the scores averaged 3.05 (95% Confidence Interval (Cl): [2.93, 3.16]). Our findings indicate that at least half of the GPT-generated themes align with those in published studies, exhibiting succinctness with minimal repetition, substantial depth of explanations, and relevant quotations. Despite these promising results, practices such as complementing outputs with field-specific knowledge are recommended.