Online dictionary learning from large-scale binary data

Yanning Shen, Georgios B. Giannakis

Abstract: Compressive sensing (CS) has been shown useful for reducing dimensionality, by exploiting signal sparsity inherent to specific domain representations of data. Traditional CS approaches represent the signal as a sparse linear combination of basis vectors from a prescribed dictionary. However, it is often impractical to presume accurate knowledge of the basis, which motivates data-driven dictionary learning. Moreover, in large-scale settings one may only afford to acquire quantized measurements, which may arrive sequentially in a streaming fashion. The present paper jointly learns the sparse signal representation and the unknown dictionary when only binary streaming measurements with possible misses are available. To this end, a novel efficient online estimator with closed-form sequential updates is put forth to recover the sparse representation, while refining the dictionary `on the fly'. Numerical tests on simulated and real data corroborate the efficacy of the novel approach.