Estimating Double Sparse Structures Over ℓ(ℓ) -Balls: Minimax Rates and Phase Transition

Zhifan Li, Yanhang Zhang, Jianxin Yin

Abstract: In this paper, we focus on the high-dimensional double sparse structures, where the parameter of interest simultaneously encourages group-wise and element-wise sparsity. By combining the Gilbert-Varshamov bound and its variants, we develop a novel lower bound technique for the metric entropy of the parameter space, specifically tailored for the double sparse structure over $\ell _{u}(\ell _{q})$ -balls with $u,q \in [0,2$ ). We give lower bounds on the estimation error using an information-theoretic approach, leveraging the proposed technique and Fano’s inequality. To complement the lower bounds, we establish matching upper bounds through a direct analysis of constrained least-squares estimators and utilizing results from empirical processes. A significant discovery is that a phase transition phenomenon exists on the minimax rates for $u,q \in (0, 2)$ . Furthermore, we extend the theoretical findings to the double sparse regression models and determine the minimax rates for estimation error. A novel Double Sparse Iterative Hard Thresholding (DSIHT) procedure is developed, with minimax optimality guaranteed. Finally, we demonstrate the superiority of the proposed method through numerical experiments.