SDGM: Sparse Bayesian Classifier Based on a Discriminative Gaussian Mixture Model

Hideaki Hayashi, Seiichi Uchida

TL;DR: A sparse classifier based on a discriminative Gaussian mixture model, which can also be embedded into a neural network.

Abstract: In probabilistic classification, a discriminative model based on Gaussian mixture exhibits flexible fitting capability. Nevertheless, it is difficult to determine the number of components. We propose a sparse classifier based on a discriminative Gaussian mixture model (GMM), which is named sparse discriminative Gaussian mixture (SDGM). In the SDGM, a GMM-based discriminative model is trained by sparse Bayesian learning. This learning algorithm improves the generalization capability by obtaining a sparse solution and automatically determines the number of components by removing redundant components. The SDGM can be embedded into neural networks (NNs) such as convolutional NNs and can be trained in an end-to-end manner. Experimental results indicated that the proposed method prevented overfitting by obtaining sparsity. Furthermore, we demonstrated that the proposed method outperformed a fully connected layer with the softmax function in certain cases when it was used as the last layer of a deep NN.

Code: https://drive.google.com/file/d/1WQJS7lZGXx7JSnje3DgmBQQx0zVi3Lqe/view?usp=sharing

Keywords: classification, sparse Bayesian learning, Gaussian mixture model