Is Safer Better? The Impact of Guardrails on the Argumentative Strength of LLMs in Hate Speech Countering.

Helena Bonaldi, Greta Damo, Nicolás Benjamín Ocampo, Elena Cabrio, Serena Villata, Marco Guerini

Abstract:

The potential effectiveness of counterspeech as a hate speech mitigation strategy is attracting
increasing interest in the NLG research community, particularly towards the task of automati-
cally producing it. However, automatically generated responses often lack the argumentative
richness which characterises expert-produced counterspeech. In this work, we focus on two
aspects of counterspeech generation to produce more cogent responses. First, by investigating
the tension between helpfulness and harmlessness of LLMs, we test whether the presence of
safety guardrails hinders the quality of the generations. Secondly, we assess whether attack-
ing a specific component of the hate speech results in a more effective argumentative strategy
to fight online hate. By conducting an extensive human and automatic evaluation, we show
how the presence of safety guardrails can be detrimental also to a task that inherently aims
at fostering positive social interactions. Moreover, our results show that attacking a specific
component of the hate speech, and in particular its implicit negative stereotype and its hateful
parts, leads to higher-quality generations.