De-Biasing The Lasso With Degrees-of-Freedom Adjustment

Pierre C Bellec, Cun-Hui Zhang

Abstract:

This paper studies schemes to de-bias the Lasso in a linear model y=Xβ+ϵ where the goal is to construct confidence intervals for aT0β in a direction a0, where X has iid N(0,Σ) rows. We show that previously analyzed propositions to de-bias the Lasso require a modification in order to enjoy efficiency in a full range of sparsity. This modification takes the form of a degrees-of-freedom adjustment that accounts for the dimension of the model selected by Lasso.
Let s0 be the true sparsity. If Σ is known and the ideal score vector proportional to XΣ−1a0 is used, the unadjusted de-biasing schemes proposed previously enjoy efficiency if s0⋘n2/3. However, if s0⋙n2/3, the unadjusted schemes cannot be efficient in certain a0: then it is necessary to modify existing procedures by a degrees-of-freedom adjustment. This modification grants asymptotic efficiency for any a0 when s0/p→0 and s0log(p/s0)/n→0.
If Σ is unknown, efficiency is granted for general a0 when
s0logpn+min{sΩlogpn,∥Σ−1a0∥1logp−−−−√∥Σ−1/2a0∥2n−−√}+min(sΩ,s0)logpn−−√→0
where sΩ=∥Σ−1a0∥0, provided that the de-biased estimate is modified with the degrees-of-freedom adjustment. The dependence in s0,sΩ and ∥Σ−1a0∥1 is optimal. Our estimated score vector provides a novel methodology to handle dense a0.
Our analysis shows that the degrees-of-freedom adjustment is not needed when the initial bias in direction a0 is small, which is granted under stringent conditions on Σ−1. The main proof argument is an interpolation path similar to that typically used to derive Slepian's lemma. It yields a new ℓ∞ error bound for the Lasso which is of independent interest.