On Sparsity and Sub-Gaussianity in the Johnson- Lindenstrauss Lemma

TMLR Paper4587 Authors

Abstract:

We provide a simple proof of the Johnson-Lindenstrauss lemma for sub-Gaussian variables.
We extend the analysis to identify how sparse projections can be, and what the cost of
sparsity is on the target dimension. The Johnson-Lindenstrauss lemma is the theoretical core
of the dimensionality reduction methods based on random projections. While its original
formulation involves matrices with Gaussian entries, the computational cost of random
projections can be drastically reduced by the use of simpler variables, especially if they
vanish with a high probability. In this paper, we propose a simple and elementary analysis
of random projections under classical assumptions that emphasizes the key role of sub-
Gaussianity. Furthermore, we show how to extend it to sparse projections, emphasizing the
limits induced by the sparsity of the data itself.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Jean_Barbier2

Submission Number: 4587