PhysicalTwin: Mixed Reality Interaction Environment for AI-supported Assistive Robots
Keywords: assistive robotics, human-robot interaction, shared user control, mixed reality, visual cues
TL;DR: PhysicalTwin - a MR testbed environment for controlling a virtual and real Kinova Jaco assistive robotic arm via different AI-supported control methods.
Abstract: Robotic arms belong to a group of innovative and powerful assistive technologies which can support people with motor impairments. These assistive devices allow people to perform Activities of Daily Living (ADLs) involving grasping and manipulating objects in their environment without caregivers' support. However, controlling a robotic arm can be a challenging and time-consuming task. Further, assistive robot technologies' often bulky, expensive, and complex nature makes developing and testing new interaction and control options a laborious and potentially precarious endeavor. To address this, we designed and developed PhysicalTwin - a Mixed Reality interaction environment for AI-supported assistive robots. PhysicalTwin contains a virtual Kinova Jaco assistive robotic arm, which can be controlled via different AI-supported control methods. The virtual Kinova Jaco can also be mirrored directly via Robot Operating System (ROS) to its real-world version for direct control. The entire MR continuum can be used to interact with the robot and receive visual feedback, resulting in a fully customizable environment for the user.
Submission Number: 20
Loading