Pareto models for discriminative multi-class linear dimensionality reduction

Karim Abou-Moustafa, Fernando De la Torre, Frank Ferrie

Abstract: We address the class masking problem in multi-class linear discriminant analysis (LDA). In the multi-class setting, LDA does not maximize each pairwise distance between classes, but rather maximizes the sum of all pairwise distances. This results in serious overlaps between classes that are close to each other in the input space, and degrades classification performance. Our research proposes Pareto Discriminant Analysis (PARDA); an approach for multi-class discriminative analysis that builds over multi-objective optimizing models. PARDA decomposes the multi-class problem to a set of objective functions, each representing the distance between every pair of classes. Unlike existing LDA extensions that maximize the sum of all distances, PARDA maximizes each pairwise distance to maximally separate all class means, while minimizing the class overlap in the lower dimensional space. Experimental results on various data sets show consistent and promising performance of PARDA when compared with well-known multi-class LDA extensions.