Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases

Rishabh Bhardwaj, Soujanya Poria

Abstract:

Red-teaming has been a widely adopted way to evaluate the harmfulness of Large
Language Models (LLMs). It aims to jailbreak a model’s safety behavior to
make it act as a helpful agent disregarding the harmfulness of the query. Existing
methods are primarily based on input text-based red-teaming such as adversarial
prompts, low-resource prompts, or contextualized prompts to condition the model
in a way to bypass its safe behavior. Bypassing the guardrails uncovers hidden
harmful information and biases in the model that are left untreated or newly introduced
by its safety training. However, prompt-based attacks fail to provide
such a diagnosis owing to their low attack success rate, and applicability to specific
models. In this paper, we present a new perspective on LLM safety research
i.e., parametric red-teaming through Unalignment. It simply (instruction) tunes
the model parameters to break model guardrails that are not deeply rooted in the
model’s behavior. Unalignment using as few as 100 examples can significantly
bypass commonly referred to as CHATGPT, to the point where it responds with
an 88% success rate to harmful queries on two safety benchmark datasets. On
open-source models such as VICUNA-7B and LLAMA-2-CHAT 7B AND 13B, it
shows an attack success rate of more than 91%. On bias evaluations, Unalignment
exposes inherent biases in safety-aligned models such as CHATGPT and LLAMA-
2-CHAT where the model’s responses are strongly biased and opinionated 64% of
the time.