SVIP: Towards Verifiable Inference of Open-source Large Language Models
Keywords: Large Language Models; Verifiable Inference; Trustworthy AI; Adversarial Attack
TL;DR: We propose SVIP, a secret-based protocol for verifiable LLM inference using intermediate outputs, allowing users to verify whether the computing provider has used the user-specified LLM with high accuracy and minimal computational cost.
Abstract: Open-source Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language understanding and generation, leading to widespread adoption across various domains. However, their increasing model sizes render local deployment impractical for individual users, pushing many to rely on decentralized computing service providers for inference through a blackbox API. This reliance introduces a new risk: a computing provider may stealthily substitute the requested LLM with a smaller, less capable model without consent from users, thereby delivering inferior outputs while benefiting from cost savings. In this paper, we formalize the problem of verifiable inference for LLMs. Existing verifiable computing solutions based on cryptographic or game-theoretic techniques are either computationally uneconomical or rest on strong assumptions. We introduce $\texttt{SVIP}$, a secret-based verifiable LLM inference protocol that leverages intermediate outputs from LLMs as unique model identifiers. By training a proxy task on these outputs and requiring the computing provider to return both the generated text and the processed intermediate outputs, users can reliably verify whether the computing provider is acting honestly. In addition, the integration of a secret mechanism further enhances the security of our protocol. We thoroughly analyze our protocol under multiple strong and adaptive adversarial scenarios. Our extensive experiments demonstrate that $\texttt{SVIP}$ is accurate, generalizable, computationally efficient, and resistant to various attacks. Notably, $\texttt{SVIP}$ achieves false negative rates below $5\\%$ and false positive rates below $3\\%$, while requiring less than $0.01$ seconds per query for verification.
Primary Area: foundation or frontier models, including LLMs
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 11602
Loading