From Shots to Narratives: Expanding Multimodal Approaches to Filmic Storytelling in the Digital Humanities

Chiao-I Tseng, Leandra Thiele, Bernhard Liebl, Eric Müller-Budack, John A. Bateman, Manuel Burghardt, Gullal S. Cheema, Ralph Ewerth

Abstract:

In recent years, digital humanities (DH) research has evolved from its textual origins to
encompass film and video studies as critical areas of inquiry as well. Nevertheless, much of
this research has remained tied to the formal levels of description most readily revealed by
automatic processing. This maintains a gap between treatments in terms of formal technical
features and the concerns of many researchers involved in film analysis of a more qualitative,
interpretative nature, thereby reiterating the classic tension within digital humanities as such:
i.e., how to relate levels of description that are ‘computable’ and those more responsive to
broader humanities-oriented interests. In this article we set out an approach to this challenge
that incorporates a multi-layered analytic framework capable of specifying increasingly abstract
descriptions in terms of patterns at lower levels. This enables us to start bringing concerns of
narrative organization and interpretation into analysis at scale. We set out the overall approach
and show several examples of its use.