Affinity and Diversity: A Unified Metric for Demonstration Selection via Internal Representations

Mariko Kato, Hakaze Cho, Yoshihiro Sakai, Naoya Inoue

Abstract: The performance of In-Context Learning (ICL) is highly sensitive to the selected demonstrations. Existing approaches to demonstration selection optimize different objectives, yielding inconsistent results. To address this, we propose a unified metric--affinity and diversity--that leverages ICL model's internal representations. Our experiments show that both affinity and diversity strongly correlate with test accuracies, indicating their effectiveness for demonstration selection. Moreover, we show that our proposed metrics align well with various previous works to unify the inconsistency.