Algorithms, Implementation, and Studies on Eating with a Shared Control Robot Arm

Laura Herlant

Abstract:

People with upper extremity disabilities are gaining increased
independence through the use of assisted devices such as
wheelchair-mounted robotic arms. However, the increased capability and
dexterity of these robotic arms also makes them challenging to control
through accessible interfaces like joysticks, sip-and-puff, and buttons that
are lower-dimensional than the control space of the robot. The potential for
robotics autonomy to ease the control burden within assistive domains has
been recognized for decades. While full autonomy is an option, it removes
all control from the user. When this is not desired by the human, the
assistive technology has, in fact, made them less able and discards useful
input the human might provide. For example, the leveraging of superior
user situational awareness to improve system robustness could be lost.

This thesis takes an in-depth dive into how to add autonomy to an
assistive robot arm in the specific application of eating, and how to make it
faster and more enjoyable for people with disabilities to feed themselves.
While we are focused on this specific application, the tools and insights we
gain can generalize to the fields of deformable object manipulation,
behavior library selection, intent prediction, robot teleoperation, and
human-robot interaction. The nature of the physical proximity and the
heavy dependence on the robot arm for doing daily tasks creates a very
high-stakes human-robot interaction.

We build the foundations for a system that is capable of fully
autonomous feeding by (1) predicting bite timing based on social cues,(2)
detecting relevant features of the food using RGBD sensor data, and (3)
automatically selecting a goal for a food-collection motion primitive to
bring a bite from the plate to the operator’s mouth. We investigate the
desired level of autonomy through user studies with an assistive robot
where users have varying degrees of control over the bite timing, control
mode-switching, and direct teleoperation of the robot to determine the
effect on cognitive load, acceptance, trust, and task performance.