Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

ICLR 2026 Conference Submission14967 Authors

Keywords: Audio, Voice agent, Trustworthiness

Abstract: With the rapid advancement and adoption of Audio Large Language Models (ALLMs), voice agents are now being deployed in high-stakes domains such as banking, customer service, and IT support. However, their vulnerabilities to adversarial misuse still remain unexplored. While prior work has examined aspects of trustworthiness in ALLMs, such as harmful content generation and hallucination, systematic security evaluations of voice agents are still lacking. To address this gap, we propose Aegis, a red-teaming framework for the governance, integrity, and security of voice agents. Aegis models the realistic deployment pipeline of voice agents and designs structured adversarial scenarios of critical risks, including privacy leakage, privilege escalation, resource abuse, etc. We evaluate the framework through case studies in banking call centers, IT Support, and logistics. Our evaluation reveals several important findings. First, restricting agents to query-based database access eliminates authentication bypass and privacy leakage attacks. However, behavioral threats such as privilege escalation, instruction poisoning, and resource abuse persist even under stricter access controls, indicating that compliance-driven vulnerabilities cannot be mitigated by data access policies alone. Moreover, open-weight models show consistently higher susceptibility to adversarial manipulation compared to closed-source ones. In addition, we also found that attacker personas and gender cues can influence outcomes but are not dominant factors when strong operational policies are enforced. These insights underscore the necessity of layered defense strategies-combining access control, policy enforcement, and behavioral monitoring- to secure next-generation ALLM-powered voice agents.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 14967