On Optimizing Large Scale Multi-Class Logistic Regression

Yifan Kang, Yarui Cao, Kai Liu

Keywords: Multinomial Logistic Regression, randomized block, Proximal Gradient, stochastic

Abstract: In this paper, we study multinomial logistic regression (MLR), a fundamental machine learning algorithm used for multi-class classification problems. We first analyze some favorable properties of the MLR objective function. By leveraging these properties, we design an optimization algorithm that operates in a feature-wise manner, which offers potential advantages in terms of computational efficiency and scalability. We also provide a convergence analysis for the proposed algorithms (both stochastic and cyclic versions). We establish theoretical guarantees that ensure the algorithm converges, thereby validating its effectiveness in optimizing the MLR model. To assess the practical performance of our algorithm, we compare our approach with a range of commonly used MLR algorithms. The experimental results demonstrate the efficiency of our algorithm.

Submission Number: 14