Moment-Based Adjustments of Statistical Inference in High-Dimensional Generalized Linear Models

Kazuma Sawaya, Yoshimasa Uematsu, Masaaki Imaizumi

Abstract: We develop a statistical inference method for generalized linear models (GLMs) in high-dimensional settings, where the number of unknown coefficients $p$ is of the same order as the sample size $n$. In this regime, constructing confidence intervals requires estimating unknown hyperparameters, such as the signal strength. However, existing estimators for the hyperparameters are not stably applicable to GLMs when $p/n$ is close to or greater than $1$, both theoretically and empirically. In this study, we develop an estimator for the hyperparameter that addresses the issue and establish an inferential framework, provided that the link function of the GLM exhibits an asymmetry property. The proposed estimator utilizes the moments of the output variable of GLMs and a convex surrogate loss. Our framework is theoretically valid even when the limit of $p/n$ exceeds $1$, ensuring the strong consistency of the hyperparameter estimator and asymptotically attaining the exact coverage probability of the confidence intervals. Our numerical experiments support these theoretical results.