pSAE-chiatry: Utilizing Sparse Autoencoders to Uncover Mental-Health-Related Features in Language Models

Declan Grabb

Keywords: mental health, psychiatry, interpretability

Abstract: As AI-powered mental health chatbots become more prevalent, their inability to recognize and respond to psychiatric emergencies, such as suicidality and mania, raises significant safety concerns. This study explores the internal representations of mental-health-related features (MHRF) in the Gemma-2-2B language model, focusing on crises related to suicide, mania, and psychosis. Using a sparse autoencoder (GemmaScope-RES-16K11) and psychiatric expertise (from M.D. mental health clinicians), MHRF's were identified across all 25 layers of the model, finding 29 features related to suicide and 42 to sadness. However, no features related to mania or paranoia were identified, suggesting critical gaps in the model’s ability to handle complex psychiatric symptoms. One feature pertaining to "suicide" was selected for further, directed study. Four prompts (two pertaining to homicide, two pertaining to suicide) were tested to evaluate the associated activations of this particular "suicide" feature, and this feature was preferentially activated by prompts pertaining to suicide, supporting the relevance of the identified features. Lastly, as proof-of-concept, steering Gemma-2-2B through enhancement of this "suicide" feature causally impacted model behavior, making Gemma-2-2B far more likely to discuss concepts related to suicide. These findings underscore the need for improved feature identification and modulation within AI models to enhance their safety and effectiveness in mental healthcare applications. Trigger warning: This work contains references to suicide.

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 12532