On-Device Interaction Mining

Deniz Arsan, Carl Guo, Rizky Wellyanto, Erik Ji, Jerry O. Talton III, Ranjitha Kumar

Abstract: Interaction mining is a popular technique for capturing design and interaction data while a mobile app is being used. Over the years, researchers have leveraged interaction mining systems to build large repositories of interaction data, enabling novel, ML-based tools for UX practitioners, designers, and programmers. Existing interaction mining systems range from simple screen recorders — which are easy to use but capture sparse, unstructured data — to complex installations requiring dedicated hardware and custom OS forks — which yield rich, high-fidelity traces but are difficult to deploy outside of a lab environment. This paper presents ODIM, an on-device framework for mobile interaction mining that produces detailed trace metadata. The framework is reified in an Android implementation based on a simple APK that users can install on their personal devices. The paper outlines ODIM’s design principles, describes its implementation, evaluates the system on traces collected from 100 popular apps on the Google Play Store, and discusses future avenues for scaling the utility and impact of interaction mining systems. The ODIM software, source code, and online trace repository are all freely available at interactionmining.org, for anyone to use and contribute to.