Label Refining: a semi-supervised method to extract voice characteristics without ground truth

Mathias Quillot, Richard Dufour, Jean-français Bonastre

Keywords: characteristics, characteristics extraction, characteristics evaluation, voice characteristics, label refining, refined labels, semi-supervision

Abstract: A characteristic is a distinctive trait shared by a group of observations which may be used to identify them. In the context of voice casting for audiovisual productions, characteristic extraction has an important role since it can help explaining the decisions of a voice recommendation system, or give modalities to the user with the aim to express a voice search request. Unfortunately, the lack of standard taxonomy to describe comedian voices prevents the implementation of an annotation protocol. To address this problem, we propose a new semi-supervised learning method entitled Label Refining that consists in extracting refined labels (e.g. vocal characteristics) from known initial labels (e.g. character played in a recording). Our proposed method first suggests using a representation extractor based on the initial labels, then computing refined labels using a clustering algorithm to finally train a refined representation extractor. The method is validated by applying Label Refining on recordings from the video game MassEffect 3. Experiments show that, using a subsidiary corpus, it is possible to bring out interesting voice characteristics without any a priori knowledge.

One-sentence Summary: We propose in this work a new semi-supervised method entitled Label Refining that consists in extracting characteristics without a priori knowledge.