Markov Decision Process for imbalanced classification

Chunyu Xuan, Jing Yang, Zhou Jiang, Dong Zhang

Abstract: In imbalanced classification problems, the samples of different classes are so imbalanced that the model cannot effectively identify the minority class samples. To solve this problem, this article proposes a new algorithm which is named TargetValue algorithm. It constructs a Markov Decision Process according to the imbalanced data set. And the reward function is carefully designed. Since the constructed Markov Decision Process has simple dynamics, the action value function can be directly calculated by derivation and handed over to the neural network for fitting. The neural network classifies unknown samples by comparing the values of different action. This article analyzes the reasons for the effectiveness of the algorithm from two perspectives: the reward function influence both the target value and the gradient of the long-term expected return. And binary classification and multi-classification experiments on multiple imbalanced data sets are conducted to verify the effectiveness of the algorithm.