Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluation

ACL ARR 2025 May Submission4309 Authors

Abstract: Suicide remains a major global mental health challenge, and early intervention hinges on recognizing signs of suicidal ideation. In private conversations, such ideation is often expressed in subtle or conflicted ways, making detection especially difficult. Existing datasets primarily rely on public help-seeking platforms like Reddit, which fail to capture the introspective and ambiguous nature of suicidal ideation in more private contexts. To address this gap, we introduce DeepSuiMind, a novel dataset of 1,200 test cases simulating implicit suicidal ideation within psychologically rich dialogue scenarios. Each case is grounded in psychological theory, combining the Death/Suicide Implicit Association Test (D/S-IAT) patterns, expanded suicidal expressions, cognitive distortions, and contextual stressors. We further propose a psychology-guided evaluation framework to assess the ability of LLMs to identify implicit suicidal ideation through their responses. Experiments with eight widely used LLMs across varied prompting conditions reveal that current models often struggle significantly to recognize implicit suicidal ideation. Our findings highlight the urgent need for more clinically grounded evaluation frameworks and design practices to ensure the safe use of LLMs in sensitive support systems.

Paper Type: Long

Research Area: Human-Centered NLP

Research Area Keywords: Identify Implicit Suicide Ideation, LLM Evaluation, Implicit Association Test

Contribution Types: Data resources

Languages Studied: English

Submission Number: 4309