Probabilistic Differentiable Filters Enable Ubiquitous Robot Control with Smartwatches

Fabian C Weigend, Xiao Liu, Heni Ben Amor

Keywords: Ubiquitous Robot Control, Differentiable Bayesian Filters

TL;DR: We propose the utilization of a differentiable filter for control using smart devices, which enables less-constrained movements while maintaining stable and effective human pose estimations, making it suitable for human-robot collaboration.

Abstract: Ubiquitous robot control and human-robot collaboration using smart devices poses a challenging problem primarily due to strict accuracy requirements and sparse information. This paper presents a novel approach that incorporates a probabilistic differentiable filter, specifically the Differentiable Ensemble Kalman Filter (DEnKF), to facilitate robot control solely using Inertial Measurement Units (IMUs) from a smartwatch and a smartphone. The implemented system is cost-effective and achieves accurate estimation of the human pose state. Experiment results from human-robot handover tasks underscore that smart devices allow versatile and ubiquitous robot control. The code for this paper is available at https://github.com/ir-lab/DEnKF and https://github.com/wearable-motion-capture.

Submission Number: 2