Demonstrating XDTK: Prototyping Multi-Device Interaction and Arbitration in XR

Eric J. Gonzalez, Ishan Chatterjee, Khushman Patel, Mar González-Franco, Andrea Colaço, Karan Ahuja

Abstract: The interaction space of XR head-mounted devices can be extended by leveraging other digital devices, such as phones, tablets, and smartwatches. We present a demonstration of XDTK (Cross-Device Toolkit), an open-sourced prototyping toolkit for multi-device interactions in XR. The toolkit consists of two parts: (1) an Android app that runs on client devices and surfaces pose, touch, and other sensor data to a (2) Unity server that can be added to any Unity-based XR application. For this demo, we specifically apply XDTK toward a few example applications, including multi-device arbitration. By leveraging pose data from each device, we can infer which device the user is gazing at to seamlessly hand off control and display between multiple devices. We also show examples leveraging a tablet sketching and a smartwatch for menu navigation.