Towards Robot Avatars: Systems and Methods for Teleinteraction at Avatar XPRIZE Semi-Finals

Rui Luo, Chunpeng Wang, Eric Schwarm, Colin Keil, Evelyn Mendoza, Pushyami Kaveti, Stephen Alt, Hanumant Singh, Taskin Padir, John Peter Whitney

Abstract: There has been a drastic shift to remote interaction for professional, industrial and personal interactions. Improving the overall quality of these interactions by removing any sense of distance between the users is the ultimate goal. Video conferencing has been widely adopted as an improvement to audio-only interactions. Having added visuals to audio communication, the next frontier is to add physical interaction to this remote communication. In this paper, we present an avatar system with the aim of tackling these necessities. The proposed system includes both hardware and software designs to ensure a real-time telemanipulation experience with tactile force feedback. We present a coupled hydrostatic actuated gripper and glove with high system bandwidth to reduce the inherent latency of the mechanical system. To account for latency over the network, the wave variable based method is adopted to maintain the stability of the closed-loop gripper control even under hundreds of milliseconds of delay. A bidirectional audiovisual communication system comprised of off-the-shelf hardware and software is incorporated to allow realtime conversation between the operator and the recipient for collaborative tasks. the proposed system has been validated in lab experiments and the global ana avatar xprize challenge semifinal.