Do cues in a video help in handling rare words in a machine translation system under a low-resource setting?

Download PDF
Open Webpage

Loitongbam Sanayai Meetei, Alok Singh, Thoudam Doren Singh, Sivaji Bandyopadhyay

Published: 2023, Last Modified: 06 Feb 2025Nat. Lang. Process. J. 2023EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: Highlights•Enhance the performance of a machine translation system under a low resource setting.•Build the first synthetic English-Hindi multimodal dataset with video modality.•Identify the encoding technique that captures the key information in a video.•Extensive analysis of text-only NMT and attention-based video-guided MMT systems.•Demonstrate the significance of video cues in a machine translation system.