Automatic assessment of communication skill in real-world job interviews: A comparative study using deep learning and domain adaptation

Jinal Hitesh Thakkar, Chinchu Thomas, Dinesh Babu Jayagopi

Abstract: With the increasing use of video-based job interviews, there is a growing demand for automated tools that can accurately evaluate the interviewee’s performance. While hiring decisions have traditionally been made based on a combination of a candidate’s knowledge, skills, and overall fit for the job, the use of automated evaluation tools can help provide more objective and consistent assessments. Traditional methods of assessing communication skills are subjective and prone to biases, leading to inconsistent evaluation of candidates. Such tools can assist in identifying candidates who possess the necessary competencies for the job and may help streamline the recruitment process for employers. This paper explores the use of deep learning and domain adaptation algorithms to improve the automatic assessment of communication skills for candidates preparing for the Civil Services Examination. Automated assessment tools provide a more objective and consistent evaluation, based on speech, language, and nonverbal cues in video-based interviews. However, these tools require large amounts of labeled data, which can be a challenge in some cases. To address this issue, we have used domain adaptation, which can transfer knowledge from a source domain to improve predictions in a target domain. The paper investigates video classification and multimodal classification methods, attention mechanisms, and deep learning techniques, along with domain adaptation approaches to enhance performance. The paper presents the interview dataset collected in an uncontrolled setting, our modeling approaches adopted to address the problem, and the experimental results and analysis. The results show the best classification accuracy is 71%.