Modeling Winner-Take-All Competition in Sparse Binary Projections

Wenye Li

Keywords: Sparse Representation, Sparse Binary Projection, Winner-Take-All

TL;DR: We developed a Winner-Take-All model that learns a sparse binary representation for input samples, with significantly improved speed and accuracy.

Abstract: Inspired by the advances in biological science, the study of sparse binary projection models has attracted considerable recent research attention. The models project dense input samples into a higher-dimensional space and output sparse binary data representations after Winner-Take-All competition, subject to the constraint that the projection matrix is also sparse and binary. Following the work along this line, we developed a supervised-WTA model when training samples with both input and output representations are available, from which the optimal projection matrix can be obtained with a simple, efficient yet effective algorithm. We further extended the model and the algorithm to an unsupervised setting where only the input representation of the samples is available. In a series of empirical evaluation on similarity search tasks, the proposed models reported significantly improved results over the state-of-the-art methods in both search accuracy and running time. The successful results give us strong confidence that the work provides a highly practical tool to real world applications.