Developing a closed captioning quality assessment system using a multi-label classifier with active learning from deaf and hard of hearing viewers

Somang Nam, Deborah I. Fels, Mark H. Chignell

Abstract: Closed Captioning (CC) is a telecommunications service to display textual information equivalent to audio. Although the primary consumer group is Deaf (D) and Hard of Hearing (HOH) viewers, they are typically excluded from the quality assessment process. Including D and HOH viewers for all assessments is nearly impossible and requires enormous effort. One solution is to use machine learning algorithms to replicate human subjective evaluation of the quality of CC. In this paper, a multi-label classifier was trained using an active learning algorithm with D and HOH viewers, predicting human subjective ratings on the various quality of CC. An online user study was conducted to train the multilayer perception with D and HoH participant subjective quality assessment rating data on machine-queried CC encoded to short video clips in various genres. The result revealed the possibilities of using an automated assessment system to predict the human perceived quality of CC, human preferences and perceiving behaviour upon viewing CC with deliberate error inclusion.

External IDs:dblp:journals/apin/NamFC23