Weekend at BERTie's : A Hierarchical Encoding for Dialog Act Classification

Rayane Mouhli, Thibaut Valour

Abstract: Conversational agents like Siri, Alexa and ChatGPT have gained immense popularity due to their ability to comprehend the type of information being conveyed by the user and generate appropriate responses based on context. This task is known as Dialog Act Classification. In this paper, we propose a hierarchical encoding strategy to tackle this problem. We use a BERT model to encode each utterance of a dialog, then, a BiLSTM encodes the encoded utterances from a same dialog. Since we encode at the utterance-level then at the dialog level, our model has a well-understanding of the context of a sentence to classify it. We conducted our experiments on the dyda_da dataset from the SILICONE benchmark developed by HuggingFace, which contains everyday communication styles and diverse topics related to daily life. Our model outperforms the baseline BERT model, achieving an accuracy of 85% on the validation set. We also analyse the influence of context and imbalanced data on the performance of the model.