Unsupervised Script Generation from Narrations of Instructional Videos

Anonymous

Abstract: This work explores the problem of generating scripts of real-world activities. Different from prior formulations, we consider a setting where text transcripts of instructional videos performing a real-world activity (e.g., making coffee) are provided and the goal is to identify the key steps relevant to the task as well as the dependency relationship between these key steps. We propose a novel script generation approach that combines the reasoning capabilities of instruction-tuned language models along with clustering and ranking components to generate accurate scripts in a completely unsupervised manner. We show that the proposed approach generates more accurate scripts compared to a supervised script learning approach on tasks from the Procel and Crosstask datasets.

Paper Type: short

Research Area: NLP Applications