Uncovering Safety Risks of Large Language Models through Concept Activation Vector

Zhihao Xu, Ruixuan HUANG, Changyu Chen, Xiting Wang

Keywords: large language model, responsible AI, AI safety, concept-based model explanation

TL;DR: We introduce a concept-based attack method using safety concept activation vectors (SCAVs) to efficiently attack well-aligned LLMs, revealing potential societal risks even after safety alignment.

Abstract: Despite careful safety alignment, current large language models (LLMs) remain vulnerable to various attacks. To further unveil the safety risks of LLMs, we introduce a Safety Concept Activation Vector (SCAV) framework, which effectively guides the attacks by accurately interpreting LLMs' safety mechanisms. We then develop an SCAV-guided attack method that can generate both attack prompts and embedding-level attacks with automatically selected perturbation hyperparameters. Both automatic and human evaluations demonstrate that our attack method significantly improves the attack success rate and response quality while requiring less training data. Additionally, we find that our generated attack prompts may be transferable to GPT-4, and the embedding-level attacks may also be transferred to other white-box LLMs whose parameters are known. Our experiments further uncover the safety risks present in current LLMs. For example, in our evaluation of seven open-source LLMs, we observe an average attack success rate of 99.14%, based on the classic keyword-matching criterion. Finally, we provide insights into the safety mechanism of LLMs. The code is available at https://github.com/SproutNan/AI-Safety_SCAV.

Primary Area: Safety in machine learning

Flagged For Ethics Review: true

Submission Number: 7016