Smart subtitles for language learning

Geza Kovacs

Abstract: Language learners often use subtitled videos to help them learn the language. However, standard subtitles are suboptimal for vocabulary acquisition, as translations are nonliteral and made at the phrase level, making it hard to find connections between the subtitle text and the words in the video. This paper presents Smart Subtitles, which are interactive subtitles tailored towards vocabulary acquisition. They provide features such as vocabulary definitions on hover, and dialog-based video navigation. Our user study shows that Chinese learners learn over twice as much vocabulary with Smart Subtitles than with dual Chinese-English subtitles, with similar levels of comprehension and enjoyment.