Non-parametric Inference Adaptive to Intrinsic Dimension

Khashayar Khosravi, Greg Lewis, Vasilis Syrgkanis

Keywords: non-parametric statistics, inference, intrinsic dimension, conditional moment equation

TL;DR: Estimation and inference results for solving non-parametric conditional moment equations with rates depending only on intrinsic dimension

Abstract: We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension $D$ of the conditioning variable is larger than the sample size $n$, estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic dimension $d$, as measured by locally low doubling measures. Our estimation is based on a sub-sampled ensemble of the $k$-nearest neighbors ($k$-NN) $Z$-estimator. We show that if the intrinsic dimension of the covariate distribution is equal to $d$, then the finite sample estimation error of our estimator is of order $n^{-1/(d+2)}$ and our estimate is $n^{1/(d+2)}$-asymptotically normal, irrespective of $D$. The sub-sampling size required for achieving these results depends on the unknown intrinsic dimension $d$. We propose an adaptive data-driven approach for choosing this parameter and prove that it achieves the desired rates. We discuss extensions and applications to heterogeneous treatment effect estimation.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1901.03719/code)