Sensing structure in learning-based binary classification of high-dimensional data

Burkay Orten, Prakash Ishwar, William Clement Karl, Venkatesh Saligrama

Abstract: In many problems of engineering interest, one encounters the situation where the ambient dimension of the sensed data is very large relative to the number of samples, but there exists a latent low dimensional sensing structure that can potentially be leveraged for inferencing tasks. This work investigates the impact of latent sensing structure on supervised classification performance when the data dimension scales to infinity faster than the number of samples. In contrast to related studies, here the classification difficulty is held fixed as the data dimension scales. For a binary supervised classification problem with Gaussian likelihood functions, it is shown that the asymptotic error probability converges to half if the sensing structure is totally ignored, whereas it converges to the Bayes risk if the sensing structure is sufficiently regular and it is properly utilized. It is also shown, however, that without suitable regularity in the latent low-dimensional sensing structure, it is impossible to attain an asymptotic error probability which is better than half. These findings are validated through various simulations. Additional numerical results for support vector machines and sensitivity to mismatch between true and assumed structure are also provided.