The Sparse Matrix-Based Random Projection: A Mean Absolute Deviation Analysis for Sparse Ternary Data

TMLR Paper6976 Authors

Abstract: In this paper, we investigate random projections based on sparse $\{0,\pm1\}$ matrices, which take sparse $\{0,\pm\mu\}$-ternary data as input. Such sparse ternary data, including $\{\pm\mu\}$-binary data as a special case, are widely used in machine learning, particularly for data quantization tasks, where they often match or even outperform their full-precision counterparts. For the projection of such ternary data, we analyze the mean absolute deviation (MAD), a metric that quantifies the dispersion of projected data points. In general, greater dispersion is expected to better capture the intrinsic variations in the original data, making it favorable for downstream classification tasks. Our analysis demonstrates that extremely sparse $\{0,\pm1\}$ matrices, such as those with only one or a few nonzero entries per row, can achieve large MAD values. By employing such sparse matrices, we indeed obtain favorable classification performance on the projected data. These highly sparse matrix structures suggest that substantial computational savings can be realized in random projection.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Lijun_Zhang1

Submission Number: 6976