Towards Open Intent Detection

Hanlei Zhang, Hua Xu, Shaojie Zhao, Qianrui Zhou

Abstract: Open intent detection is a significant problem in natural language understanding, which aims to detect the unseen open intent with the prior knowledge of only known intents. Current methods have two core challenges in this task. On the one hand, they have limitations in learning friendly representations to detect the open intent. On the other hand, there lacks an effective approach to obtaining specific and compact decision boundaries for known intents. To address these issues, this paper introduces an original framework, DA-ADB, which successively learns distance-aware intent representations and adaptive decision boundaries for open intent detection. Specifically, we first leverage distance information to enhance the distinguishing capability of the intent representations. Then, we design a novel loss function to obtain appropriate decision boundaries by balancing both empirical and open space risks. Extensive experiments show the effectiveness of distance-aware and boundary learning strategies. Compared with the state-of-the-art methods, our method achieves substantial improvements on three benchmark datasets. It also yields robust performance with different proportions of labeled data and known categories. The full data and codes are available at https://github.com/thuiar/TEXTOIR