Leveraging Generative Mode-Seeking for Precision Matrix Estimation

Keshav Goyal, Suraj yadav

Keywords: Precision Matrix Estimation, Data Augmentation, Generative Modelling, Diffusion Models

Abstract: While modern generative models produce abundant synthetic data, in this paper, we show that employing such data within a standard data augmentation framework provides no additional value for precision matrix estimation. We demonstrate that this limitation arises from the generative models' inherent mode-seeking bias, which captures high-variance features while severely compressing low-variance components. However, we further reveal that this spectral distortion constitutes a highly structured signal rather than an uninformative artifact. We propose a novel estimator that extracts this signal by subtracting the synthetic covariance from the empirical covariance of the real data. This subtractive approach isolates the exact spectral subspace where the generative model underrepresents variance, enabling the targeted regularization of underestimated low-eigenvalue dimensions. Through theoretical analysis using idealized spectral filters and empirical evaluations, we establish our proposed method as a robust, data-driven non-linear alternative to tuned linear shrinkage baselines.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 261