Simplifying Gaussian mixture models via entropic quantization

Frank Nielsen, Vincent Garcia, Richard Nock

Abstract: Mixture models are a crucial statistical modeling tool at the heart of many challenging applications in computer vision, machine learning, and text classification among others. In this paper, we describe a novel and efficient algorithm for simplifying Gaussian mixture models using a generalization of the celebrated k-means quantization algorithm tailored to relative entropy in statistical distribution spaces. Our algorithm extends easily to arbitrary mixture of exponential families. The proposed method is shown to compare favourably well with the state-of-the-art unscented transform clustering algorithm both in terms of time and quality performances.