Learning a State Estimator for Tactile In-Hand Manipulation

Lennart Röstel, Johannes Pitz, Leon Sievers, Berthold Bäuml

Abstract:

We study the problem of estimating the pose of an
object which is being manipulated by a multi-fingered robotic
hand by only using proprioceptive feedback. To address this
challenging problem, we propose a novel variant of differentiable particle filters, which combines two key extensions.
First, our learned proposal distribution incorporates recent
measurements in a way that mitigates weight degeneracy.
Second, the particle update works on non-euclidean manifolds
like Lie-groups, enabling learning-based pose estimation in 3D
on SE(3). We show that the method can represent the rich and
often multi-modal distributions over poses that arise in tactile
state estimation. The models are trained in simulation, but by
using domain randomization, we obtain state estimators that
can be employed for pose estimation on a real robotic hand
(equipped with joint torque sensors). Moreover, the estimator
runs fast, allowing for online usage with update rates of more
than 100 Hz on a single CPU core. We quantitatively evaluate
our method and benchmark it against other approaches in
simulation. We also show qualitative experiments on the real
torque-controlled DLR-Hand II.