The Tap Free Energy for High-Dimensional Linear Regression

Jiaze Qiu, Subhabrata Sen

Abstract: We derived a variational representation for the log-normalizing constant of the posterior distribution in Bayesian linear regression with a uniform spherical prior and an i.i.d. Gaussian design. We work under the “proportional" asymptotic regime, where the number of observations and the number of features grow at a proportional rate. This rigorously establishes the Thouless-Anderson-Palmer (TAP) approximation arising from spin glass theory, and proves a conjecture of [Krzakala et al., 2014] in the special case of the spherical prior.