It takes two for security: A Verifiable Co-Aggregation Protocol for Heterogeneous Federated Distillation

Tian Wen, Sheng Sun, Yuwei Wang, Peiyan Chen, Zhiyuan Wu, Renshuai Tao, Min Liu, Bo Gao

Keywords: Secure Aggregation, Federated Distillation, Verifiable Computing, Privacy Protection

Abstract: Federated distillation (FD) enables efficient collaboration among heterogeneous models, yet its rising application in privacy-sensitive fields raises security concerns. Advanced countermeasures have introduced secure aggregation protocols, which are broadly classified as centralized (server-dominated) or fully distributed (server-free). The former risks adversarial server interference, while the latter avoids central threats but suffers from poor energy efficiency due to coordination overhead. Neither approach offers an ideal solution. This paper introduces SVAFD, a secure co-aggregation protocol that ensures privacy protection and verifiability by redefining the responsibility boundaries between server and client in line with their respective requirements and resource capabilities. Specifically, SVAFD allows clients to perform lightweight encoding of logits and achieve locally partial aggregation, while the server converges all partial results for global decoding and generates aggregation proofs. By shifting from unilateral dominance to multilateral client/server co-aggregation, SVAFD guarantees that no vanilla clients’ privacy can be inferred while providing with an unforgeable aggregation proof for process checking, even in the presence of server collusion with a subset of clients. Moreover, SVAFD is resilient to stragglers and reduces the complexity for each aggregation from three rounds (Armadillo, NeurIPS’24) or more in recent protocols to only two rounds, chieving a ~1.32× reduction in computation consumption.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 23336