Adaptive Content Placement in Mixed Reality Through Empirical User Behavioral Patterns

Feiyu Lu, Mengyu Chen, Hsiang Hsu, Pranav Deshpande, Cheng Yao Wang, Blair MacIntyre

Abstract: Mixed Reality (MR) could assist users’ productivity work by overlaying virtual content on the user's physical environment. However, where and how to place these content to best support the users has been challenging due to the dynamic nature of MR experiences and the distinct individual characteristics of the users. Prior work mostly relies on pre-defined utilitarian values by experts to identify the importance of physical objects to the users’ tasks, which may not accurately reflect what users need and how they behave. In this work, we propose a pipeline that derive adaptive placements of MR windows by analyzing empirical user behaviors along a time horizon. By continuously observing temporal user behaviors, we could potentially achieve MR content adaptations that respects user tasks and behaviors continuously. Our pipeline consists of four modules to collect, analyze empirical user behaviors, solicit user feedback, and generate adaptations that consider empirical user behaviors.