Target speaker lipreading by audio-visual self-distillation pretraining and speaker adaptation

Jing-Xuan Zhang, Tingzhi Mao, Longjiang Guo, Jin Li, Lichen Zhang

Abstract: Highlights•Cross-lingual transfer learning enhances lipreading with limited target language data.•Speaker adaptation boosts specific speaker lipreading accuracy.•Model ensembling with lip and face inputs improves lipreading.•Our approach sets a new benchmark on the ChatCLR dataset.