The influence of pretraining model's life of orientation leaning on mental health disease detection
Abstract: With the popularity of the Internet, online social media has increasingly become a platform for people to share their attitudes towards life, such as optimism and pessimism about the future. While the Internet embraces various views, it also quietly deepens the formation of impressions on different views and attitudes. A large part of these texts will form the corpus of the pre-trained model, and the model may learn the tendency of life attitudes in the corpus. Our work develops new methods to (1) measure life attitude biases in LMs trained on such corpora and (2) measure the judgement impact of downstream models trained on different life attitude corpus. We focus on mental health disorder detection, aiming to empirically quantify the effects of life attitude (optimism, pessimism) leaning in pretraining data on the influence of risk-related tasks. Our findings reveal that pretrained LMs do have life attitude leanings that reinforce the polarization present in pretraining corpora, propagating life attitude biases into mental health disorder detection.We discussed strategies that might mitigate or leverage models of different life attitude leaning.
Paper Type: Long
Research Area: Interpretability and Analysis of Models for NLP
Research Area Keywords: contrastive explanations; data influence
Contribution Types: Model analysis & interpretability
Languages Studied: English
Submission Number: 5516
Loading