SoftmaxU: Open softmax to be aware of unknowns

Xulun Ye, Jieyu Zhao, Jiangbo Qian, Yuqi Li

Abstract: Softmax, as one of the most fundamental classification methods, has been widely exploited in the modern machine learning society. However, the conventional softmax model is trained to predict the labels in the known environment. The real world contains many unknowns (unknown classes and unknown class number). To handle this problem, first, we propose a general open softmax model (SoftmaxU). Then, to validate our proposed general open softmax framework, a deep neural network-based SoftmaxU model (DSoftmaxU) is implemented, in which Bayesian low-rank and deep non-linear subspace network is proposed to generate the unknown class number and detect the novel classes. In addition, the corresponding posterior probability inference and model optimization algorithm is derived. Finally, we demonstrate the proposed open softmax model on both the synthetic and real datasets to validate our theoretic analysis, where our model achieves an average performance improvement of 2% along with unknown class number detection against the conventional open-set, novelty detection methods. Our source code will be available on the website for the further study (https://github.com/yexlwh/SoftmaxU).