Power analysis of knockoff filters for correlated designs

Jingbo Liu, Philippe Rigollet

Abstract: The knockoff filter introduced by Barber and Cand\`es 2016 is an elegant framework for controlling the false discovery rate in variable selection. Yet, there is no conclusive result on the power (type~II error rate) analysis, or how to choose the knockoff generation method, even in the Gaussian setting. When the predictors are i.i.d.\ Gaussian, it is known that as the signal to noise ratio tend to infinity, the power of the knockoff filter tends to $1$ under any fixed FDR budget. However, when the predictors have a general covariance structure $\bsigma$, it is not obvious that one can define an analogous notion of the signal to noise ratio. We introduce the notion of \emph{effective signal deficiency} (ESD) as any functional of $\bsigma$, such that the power tend to $1$ \emph{if and only if} this functional tends to $0$ (under given noise level, sparsity ratio, and sampling rate). We then study the ESD for Lasso and the knockoff filter with different knockoff constructions, assuming the correctness of the replica method prediction for Lasso. As a baseline for comparison, we show that using Lasso with an oracle for choosing the threshold that gives the correct FDR, the ESD tends to $0$ if and only if the empirical distribution of the diagonals of the precision matrix ${\bf P}:=\bsigma^{-1}$ convergences to $0$ in distribution. In other words, the ESD can be taken as $\|(P_{jj})_{j=1}^p\|_{LP}:=\inf\left\{\epsilon>0\colon \frac{1}{p}|\{P_{jj}\ge \epsilon\}|\le \epsilon\right\}$. For the knockoff filter, if $\bf \underline{P}$ is the $2p\times 2p$ precision matrix for the predictors and knockoff variables, we show that the ESD is $\|(\underline{P}_{jj})_{j=1}^{2p}\|_{LP}$. We then find more explicit formulae for various specific knockoff constructions. We introduce the \emph{conditional independence knockoff}, which always exists for Gaussian tree graphical models (or sufficiently sparse), and show that its ESD is $\|(\Sigma_{jj}P_{jj}^2)_{j=1}^p\|_{LP}$.

CMT Num: 8941