Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety

Max Zhang, Derek Liu, Kai Zhang, Joshua Franco, Haihao Liu, Kevin Zhu

Keywords: Knowledge Distillation, Multilingual Safety, Jailbreak Prevention, Safety Degradation, Adversarial Robustness, Foundation Models, LoRA, PEFT

TL;DR: Distilling safety refusals from a proprietary teacher into open-source multilingual models via response-based knowledge distillation unexpectedly worsens jailbreak robustness, revealing hidden risks in knowledge distillation for safety alignment.

Abstract: Large language models (LLMs) are increasingly deployed worldwide, yet their safety alignment remains predominantly English-centric. This allows for vulnerabilities in non-English contexts, especially with low-resource languages. We introduce a novel application of knowledge distillation (KD) in the context of multilingual jailbreak prevention, examining its efficacy. We distill the refusal behaviors of a proprietary teacher model ($\texttt{OpenAI o1-mini}$) with Low-Rank Adaptation (LoRA) into three open-source student models: $\texttt{Meta-Llama-3-8B-Instruct}$, $\texttt{Gemma-2-2B-IT}$, and $\texttt{Qwen3-8B}$, using $\sim$28,000 multilingual jailbreak prompts from $\texttt{XSafety}$ via black-box response-based, parameter-efficient fine-tuning (PEFT). Evaluation on the $\texttt{MultiJail}$ benchmark reveals a counterintuitive behavior: standard fine-tuning on the teacher's "safe" refusal data inadvertently increases Jailbreak Success Rate (JSR) for all student models, up to 16.6 percentage points. Our experiments reveal a divergent generalization to unseen languages during distillation, with varying outcomes depending on the base model. By removing a primary source of safety degradation, nuanced 'boundary' refusals, we mitigate or even reverse safety declines in student models, although reductions in reasoning performance (GSM8K) persist. Overall, our exploratory study highlights the challenges and potential of KD as a technique for multilingual safety alignment, offering a foundation for future research in this direction. Code is available at {https://github.com/maxh119Z/RB-KD-Multilingual-Safety-Trade-offs.git}

Submission Number: 4