Evaluating Cumulative Spectral Gradient as a Complexity Measure

Haji Gul, Abdul Ghani Haji Naim, Ajaz A Bhat

Submission Track: Track 1: Machine Learning Research by Muslim Authors

Keywords: Cumulative Spectral Gap, CSG, Complexity, Knowledge Graph, Link Prediction, KGC

TL;DR: Evaluating Cumulative Spectral Gradient as a Complexity Measure

Abstract: Accurate estimation of dataset complexity is crucial for evaluating and comparing link‐prediction models for knowledge graphs (KGs). The Cumulative Spectral Gradient (CSG) metric \cite{branchaud2019spectral} —derived from probabilistic divergence between classes within a spectral clustering framework— was proposed as a dataset complexity measure that (1) naturally scales with the number of classes and (2) correlates strongly with downstream classification performance. In this work, we rigorously assess CSG’s behavior on standard knowledge‐graph link‐prediction benchmarks—a multi‐class tail‐prediction task— using two key parameters governing its computation: $M$, the number of Monte Carlo–sampled points per class, and $K$, the number of nearest neighbors in the embedding space. Contrary to the original claims, we find that (1) CSG is highly sensitive to the choice of $K$, thereby does not inherently scale with the number of target classes, and (2) CSG values exhibit weak or no correlation with established performance metrics such as mean reciprocal rank (MRR). Through experiments on FB15k‐237, WN18RR, and other standard datasets, we demonstrate that CSG’s purported stability and generalization‐predictive power break down in link‐prediction settings. Our results highlight the need for more robust, classifier-agnostic complexity measures in KG link-prediction evaluation.

Submission Number: 18