Prediction of Praising Skills Based on Multimodal Information

Toshiki Onishi, Asahi Ogushi, Ryo Ishii, Atsushi Fukayama, Akihiro Miyata

Abstract: Praising behavior is an important method of communication. An existing study constructed models to predict praising skill, which indicates the degree to which the praise is done well, by using only unimodal behavior such as speech audio or visual behavior of a praiser who gives praise in dyad interactions. To improve prediction performance, a model should be constructed that uses various additional information. In this study, we propose two approaches to predict praising skill highly accurately. The first uses trimodal (multimodal) behaviors extracted from visual, acoustic, and linguistic modalities. The second uses the behaviors of the receiver of praise since the reaction of the receiver should differ depending on how good the praise is. For this study, we collect trimodal features and the degree of praising skill in each praising scene in a dialogue. We construct multiple models to predict the degree of praising skills using various combinations of the trimodal features from the praiser and receiver. The experimental results show that the model that predicts praising skill most accurately uses multiple features related to both verbal and nonverbal behaviors of the praiser and receiver. Therefore, the two approaches of using trimodal behaviors and using features from both the receiver and praiser are effective for predicting praising skills in dyad interactions.