TeLLMate: Trusted Execution for Large Language Models at the Edge

ICLR 2026 Conference Submission14701 Authors

Keywords: Large Language Model, Security and Privacy

TL;DR: A secure and efficient framework that identifies and selectively protects the most critical layers of an LLM within a TEE.

Abstract:

Large Language Models (LLMs) are being increasingly deployed on edge devices to support real-time inference, preserve user privacy, and enhance energy efficiency. However, on-device deployment creates a lucrative attack surface for adversaries to steal the model.
Trusted Execution Environments (TEEs) provide hardware-isolated enclaves to safeguard model parameters. However, existing approaches either demand costly retraining from substantial architectural modifications or incur significant communication overhead by protecting parameters across all layers within the TEE. To address these limitations, we propose TeLLMate, which helps identify the critical layers of LLMs for TEE protection. TeLLMate delivers two key capabilities: (1) a methodology for identifying layers critical to model performance using mathematical analysis. (2) a selective protection algorithm that minimizes the secure memory footprint while defending against retraining attacks. Our experimental results demonstrate that TeLLMate offers strong protection guarantees, resulting in at least a 15x increase in perplexity and close to random-guess accuracy in zero-shot downstream tasks for the attacker’s replicated model by including at most 10% of the LLM's parameters.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 14701