Health-Conditioned Vision-Language-Action Models for Malfunction-Aware Robot Control

Huseyin ARSLAN, Ozgur Erkent

Keywords: vision-language-action models, robot health monitoring, fault-tolerant manipulation, malfunction-aware control

TL;DR: A Vision Language Action Model approach that including Health Projection in order to adapt joint malfunctions by minimal finetuning

Abstract: Research on Vision Language Action (VLA) models has been increasing rapidly in recent years. Although some of them focus on detecting, preventing and recovering from task failures, they usually don't deal with adapting to robot's physical failures. In real life scenarios, most robots face physical degradations in various ways such as joint degradation, actuator failure or weak gripper. We introduce malfunction aware (health conditioned) VLA that takes a health vector as an input that gives information about robots' joints' operation angle and torque capability, and adapts its predictions to complete the tasks with the degraded joints. To achieve this, we inject a Health Projector module to VLA-Adapter architecture and train it on malfunction robot data we collected on LIBERO environment. We collect 128 teleoperated episodes on Libero-Spatial tasks. Our results show that, with a very lightweight addition, model can learn to operate successfully with different configurations of degraded joints which default pretrained VLA-Adapter's Libero-Spatial-Pro model can not.

Submission Number: 34