Predicting Different Types of Subtle Toxicity in Unhealthy Online Conversations

Shlok Gilda, Luiz Giovanini, Mirela Silva, Daniela Oliveira

Abstract: This paper investigates the use of machine learning models to classify unhealthy online conversations containing one or more forms of subtler abuse, such as hostility, sarcasm, and generalization. We leveraged a public dataset of 44K online comments containing healthy and unhealthy comments labeled with seven forms of subtle toxicity. We were able to distinguish between these comments with a micro F1-score, macro F1-score, and ROC-AUC of 88.76%, 67.98%, and 0.71, respectively. Hostile comments were easier to detect than other types of unhealthy comments. We also conducted a sentiment analysis that revealed that most unhealthy comments were associated with a slight negative sentiment, with hostile comments being the most negative.