Workflow Graphs: A Computational Model of Collective Task Strategies for 3D Design Software

Minsuk Chang, Ben Lafreniere, Juho Kim, George Fitzmaurice, Tovi Grossman

Keywords: Software learning, Workflow analysis, Peer learning

TL;DR: We investigate how multiple demonstrations of a fixed task can be captured to automatically discover the different means of accomplishing a goal from the interaction traces of multiple users, and to encode these in a graphical representation.

Abstract: This paper introduces $\textit{Workflow graphs}$, or $\textit{W-graphs}$, which encode how the approaches taken by multiple users performing a fixed 3D design task converge and diverge from one another. The graph's nodes represent equivalent intermediate task states across users, and directed edges represent how a user moved between these states, inferred from screen recording videos, command log data, and task content history. The result is a data structure that captures alternative methods for performing sub-tasks (e.g., modeling the legs of a chair) and alternative strategies of the overall task. As a case study, we describe and exemplify a computational pipeline for building $\textit{W-graphs}$ using screen recordings, command logs, and 3D model snapshots from an instrumented version of the Tinkercad 3D modeling application, and present graphs built for two sample tasks. We also illustrate how $\textit{W-graphs}$ can facilitate novel user interfaces with scenarios in workflow feedback, on-demand task guidance, and instructor dashboards.