Classification and Representation via Separable Subspaces: Performance Limits and Algorithms

Ishan Jindal, Matthew S. Nokleby

Abstract: We study the classification performance of Kronecker-structured (K-S) subpsace models in two asymptotic regimes and develop an algorithm for fast and compact K-S subspace learning for better classification and representation of multidimensional signals by exploiting the structure in the signal. First, we study the classification performance in terms of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">diversity order</i> and pairwise geometry of the subspaces. We derive an exact expression for the diversity order as a function of the signal and subspace dimensions of a K-S model. Next, we study the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> classification capacity</i> , the maximum rate at which the number of classes can grow as the signal dimension goes to infinity. Then, we describe a fast algorithm for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Kronecker-structured learning of discriminative dictionaries</i> (K-SLD <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$</tex-math></inline-formula> ). Finally, we evaluate the empirical classification performance of K-S models for the synthetic data, showing that they agree with the diversity order analysis. We also evaluate the performance of K-SLD <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$ </tex-math></inline-formula> on synthetic and real-world datasets showing that the K-SLD <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^2$</tex-math></inline-formula> balances compact signal representation and good classification performance.