Retrieval-Free Instruction Selection for Instruction-Tuned Embedding Models via Uncentered Spectral Entropy

Abstract:

Instruction-tuned embedding models expose a consequential deployment variable: the query-side instruction.
Choosing that instruction usually requires repeated retrieval evaluation with corpus access and judged queries, precisely when such infrastructure is least available: cold-start domains, API-only embedding services, and large candidate pools that must be screened before a stable retrieval stack exists.
We study this earlier decision point and ask whether candidate instructions can be ranked before retrieval evaluation is practical.
Our central hypothesis is geometric: better instructions induce a broader, less collapsed representation geometry on a small unlabeled set of query-like proxy texts.
Based on this view, we propose Instruction Performance Prediction (IPP), a retrieval-free, label-free screening method that scores each instruction by the normalized spectral entropy of the second-moment matrix of its proxy embeddings.
Across 16 embedding models, 17 retrieval datasets, and all 272 model--dataset pairs (full $16 \times 17$ coverage), IPP attains median oriented Spearman $\rho = 0.806$ and median regret@1 of $0.004$ NDCG@10 points on a 104-instruction pool.
The same evidence also identifies a clear operating boundary: when candidate instructions produce little downstream variation, the ranking problem becomes weakly separable, so geometric screening should hand off to direct retrieval evaluation rather than be over-interpreted.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Li_Dong1

Submission Number: 8617