Index-Overlap Similarity: A Value-Free Proxy for Model Relatedness

ICLR 2026 Conference Submission19224 Authors

Keywords: machine learning, distributed learning, similarity and search, Value-Free Similarity

TL;DR: We introduce IOS, a value-free index-overlap similarity that replaces cosine to enable accurate client clustering/personalization in FL while slashing communication and leakage.

Abstract: Measuring client relatedness is central to clustering and personalization in federated learning (FL), but value-based similarities over full weights or gradients are bandwidth-heavy and leak information. We propose \emph{Index-Overlap Similarity (IOS)}, a value-free metric that represents each client by the indices of its Top-$K$ salient parameters and scores pairs by the normalized overlap of these supports. We show why IOS preserves alignment: under head-dominance with bounded dispersion, it lower-bounds cosine up to tail error; Top-$K$ is invariant to common layerwise rescalings; and exponential moving averages stabilize supports across rounds. We instantiate IOS for clustered personalized FL, neighbor selection, donor ranking, and oracle distribution alignment. Across FMNIST, CIFAR-10/100, and 20News under Dirichlet and pathological splits, IOS matches or exceeds cosine/Euclidean while sharing only indices. IOS is a simple, scalable primitive for similarity search under communication and privacy constraints.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 19224