LAVA: Long-horizon Visual Action based Food Acquisition

Amisha Bhaskar, Rui Liu, Guangyao Shi, Pratap Tokekar

Keywords: Robotic Assistive Feeding, Long-horizontal manipulation, Deformable object manipulation, Human robot interaction, Needs and seeds for society, industry, and home

TL;DR: This paper presents LAVA, a hierarchical policy framework for robotic assisted feeding, demonstrating improved adaptability and efficiency in handling a wide range of food types through sequential planning and execution.

Abstract:

Robotic Assisted Feeding (RAF) addresses the
fundamental need for individuals with mobility impairments to
regain autonomy in feeding themselves. The goal of RAF is to
use a robot arm to acquire and transfer food to individuals from
the table. Existing RAF methods primarily focus on solid foods,
leaving a gap in manipulation strategies for semi-solid and
deformable foods. This study introduces Long-horizon Visual
Action (LAVA) based food acquisition of liquid, semisolid,
and deformable foods. Long-horizon refers to the goal of
“clearing the bowl” by sequentially acquiring the food from
the bowl. LAVA employs a hierarchical policy for long-horizon
food acquisition tasks. The framework uses high-level policy
to determine primitives based on food types. At the mid-
level, LAVA finds parameters for primitives using vision. To
carry out sequential plans in the real world, LAVA delegates
action execution which is driven by Low-level policy that uses
parameters received from mid-level policy and behavior cloning
ensuring precise trajectory execution. We validate our approach
on complex real-world acquisition trials involving granular,
liquid, semisolid, and deformable food types along with fruit
chunks and soup acquisition. Across 46 bowls, LAVA acquires
much more efficiently than baselines with a success rate of
89 ± 4%, and generalizes across realistic plate variations such
as different positions, varieties, and amount of food in the bowl.
Code, datasets, videos, and supplementary materials can be
found on our website.

Video Publication Concent: (Recommended) We consent that the video will be uploaded to the workshop's YouTube channel when the submission is accepted.

Submission Number: 3