INSIGHT: An AR-Enabled User Interface for Vision-Based Markerless Interaction with IoT Nodes

Ming Xia, Yunhan Wang, Xiaoyan Wang, Zhen Cheng, Kaikai Chi

Abstract: Real-world, long-running Internet of Things (IoT) requires intense user-node interaction in the deployment, network operation, and maintenance stages. The rapidly increasing number of IoT nodes urges a new user-friendly interface to reduce the interaction complexity. This paper presents INSIGHT, an AR-enabled user interface for IoT nodes allowing the users to directly grasp perceptual node information from the surrounding videos shot by mobile AR devices such as smart glasses and phones. INSIGHT incorporates camera, magnetic, and IMU sensor data from an AR device to detect, locate, and recognize IoT nodes markerlessly. An intuitive user interface will then be overlaid on each recognized node for fetching/feeding information from/to the node. We have validated the performance of INSIGHT in terms of localization accuracy and the capability to distinguish nearby nodes. The results show that INSIGHT works well even in densely deployed networks.