Siamese vs Non-Siamese: Dual Encoders for Intent Detection

Anonymous

Abstract: Bi-encoders have been shown to be effective for intent classification. Current Bi-encoders use the same weights to learn the embedding of both the contexts and candidates. However, this can be counter-productive when there exist contexts with overlapping keywords from competing candidate labels. This could lead to unrelated context and candidate having similar embeddings and being mis-classified. In this work, we investigate the potential of non-siamese Bi-encoders for intent detection, where separate weights are learned for context and candidate. Our results show that non-siamese Bi-encoders improve the performance of traditional Bi-encoders across datasets. We also show that using heterogeneous architectures in a non-siamese Bi-encoder can effectively reduce memory and computation requirement while maintaining prediction performance.

Paper Type: short