Video as Text: A New Paradigm for Flexible Video Analysis

Li-Yun Wang, Wu-chi Feng

Abstract: The ability to capture and distribute digital videos has been available for many years. Users can easily capture high-quality video streams with mobile devices and distribute them to end users through varying platforms. This paper presents the design and implementation of a new multimedia framework called Video as Text (vText), which analyzes and manipulates video data as trivially as we handle text data in most Unix and Linux systems. In most Unix systems, it is easy to accomplish highly complex textual analysis and processing by combining relatively simple programs (e.g., grep, awk, sed, and cut) through Unix pipes. The vText paradigm seeks to mimic such programs. We demonstrate the design and implementation of vText linking video codecs with computer vision and image-processing algorithms. The experimental results indicate that the combination of simple programs provides high flexibility to users but does not incur high overhead when processing the video data.