Equivalence of State Equations from Different Methods in High-dimensional Regression

Saidi Luo, Songtao Tian, Qian Lin

Keywords: Approximate message passing, Lasso, High dimensional statistics

Abstract: State equations were firstly introduced in the approximate message passing (AMP) to describe the mean square error (MSE) in compressed sensing. Since then a set of state equations have appeared in studies of logistic regression, robust estimator and other high-dimensional statistics problems. Recently, a convex Gaussian min-max theorem(CGMT) approach was proposed to study high-dimensional statistic problems accompanying with another set of different state equations. This Paper provides a uniform viewpoint on these methods and shows the equivalence of their reduction forms, which causes that the resulting SE are essentially equivalent and can be converted into the same expression through parameter transformations. Combining these results, we show that these different state equations are derived from several equivalent reduction forms. We believe this equivalence shed light on discovering a deeper structure in high dimensional statistics.

One-sentence Summary: We showed that for some specific problem, different set of equations derived from AMP, CGMT and LOO are equivalent.