FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes

Rajat Kumar Jenamani, Priya Sundaresan, Maram Sakr, Tapomayukh Bhattacharjee, Dorsa Sadigh

Abstract:

Robot-assisted feeding has the potential to improve
the quality of life for individuals with mobility limitations who are
unable to feed themselves independently. However, there exists
a large gap between the homogeneous, curated plates existing
feeding systems can handle, and truly in-the-wild meals. Feeding
realistic plates is immensely challenging due to the sheer range of
food items that a robot may encounter, each requiring specialized
manipulation strategies which must be sequenced over a long
horizon to feed an entire meal. An assistive feeding system should
not only be able to sequence different strategies efficiently in order
to feed an entire meal, but also be mindful of user preferences
given the personalized nature of the task. We address this with
FLAIR, a system for long-horizon feeding which leverages the
commonsense and few-shot reasoning capabilities of foundation
models, along with a library of parameterized skills, to plan and
execute user-preferred and efficient bite sequences. In real-world
evaluations across 6 realistic plates, we find that FLAIR can
effectively tap into a varied library of skills for efficient food
pickup, while adhering to the diverse preferences of 42 partici-
pants without mobility limitations as evaluated in a user study.
We demonstrate the seamless integration of FLAIR with existing
bite transfer methods, and deploy it across 2 institutions
and 3 robots, illustrating its adaptability. Finally, we illustrate
the real-world efficacy of our system by successfully feeding a
care recipient with severe mobility limitations.