Hankel based maximum margin classifiers: A connection between machine learning and Wiener systems identification

Fei Xiong, Yongfang Cheng, Octavia I. Camps, Mario Sznaier, Constantino M. Lagoa

Abstract: This paper considers the problem of nonparametric identification of Wiener systems in cases where there is no a-priori available information on the dimension of the output of the linear dynamics. Thus, it can be considered as a generalization to the case of dynamical systems of non-linear manifold embedding methods recently proposed in the machine learning community. A salient feature of this framework is its ability to exploit both positive and negative examples, as opposed to classical identification techniques where usually only data known to have been produced by the unknown system is used. The main result of the paper shows that while in principle this approach leads to challenging non-convex optimization problems, tractable convex relaxations can be obtained by exploiting a combination of recent developments in polynomial optimization and matrix rank minimization. Further, since the resulting algorithm is based on identifying kernels, it uses only information about the covariance matrix of the observed data (as opposed to the data itself). Thus, it can comfortably handle cases such as those arising in computer vision applications where the dimension of the output space is very large (since each data point is a frame from a video sequence with thousands of pixels).