TransLinkGuard: Safeguarding Transformer Models Against Model Stealing in Edge Deployment

Qinfeng Li, Zhiqiang Shen, Zhenghan Qin, Yangfan Xie, Xuhong Zhang, Tianyu Du, Sheng Cheng, Xun Wang, Jianwei Yin

Abstract:

Proprietary large language models (LLMs) have been widely applied in various scenarios. Additionally, deploying LLMs on edge devices is trending for efficiency and privacy reasons. However, edge deployment of proprietary LLMs introduces new security challenges: edge-deployed models are exposed as white-box accessible to users, enabling adversaries to conduct effective model stealing (MS) attacks. Unfortunately, existing defense mechanisms fail to provide effective protection. Specifically, we identify four critical protection properties that existing methods fail to simultaneously satisfy: (1) maintaining protection after a model is physically copied; (2) authorizing model access at request level; (3) safeguarding runtime reverse engineering; (4) achieving high security with negligible runtime overhead. To address the above issues, we propose TransLinkGuard, a plug-and-play model protection approach against model stealing on edge devices. The core part of TransLinkGuard is a lightweight authorization module residing in a secure environment, e.g., TEE. The authorization module can freshly authorize each request based on its input. Extensive experiments show that TransLinkGuard achieves the same security protection as the black-box security guarantees with negligible overhead.

Primary Subject Area: [Generation] Social Aspects of Generative AI

Secondary Subject Area: [Generation] Social Aspects of Generative AI

Relevance To Conference:

This study presents a notable advancement in the intellectual property protection of edge-deployed transformer models, which serve as a universal framework for large models, making the protection of transformer models meaningful for both unimodal and multimodal situations. Specifically, as hardware advances and privacy concerns escalate, an increasing number of transformer models are being deployed at the edge. However, protecting these models poses a significant challenge, as edge-deployed models become white-box accessible to users.
In this work, we systematically identify four critical protection requirements that existing methods fail to satisfy simultaneously. To address these requirements, we introduce TransLinkGuard, a pioneering approach to intellectual property protection that effectively safeguards models based on the transformer architecture. Extensive experiments demonstrate the effectiveness of TransLinkGuard across different representative transformer models such as LLaMa2, GPT-2, and ChatGLM. Thus, by protecting transformer models, our work significantly fortifies the security of generative AI, making a notable contribution to the field.

Submission Number: 1910