CRAFT: A Neuro-Symbolic Framework for Visual Functional Affordance Grounding

Zhou Chen, Joe Lin, Sathyanarayanan N. Aakur

Keywords: Functional affordance grounding, Neuro-symbolic reasoning, Embodied AI

TL;DR: We propose a neuro-symbolic framework for functional affordance grounding that combines symbolic priors with neural visual grounding to identify action-relevant objects in cluttered scenes.

Abstract: We introduce CRAFT, a neuro-symbolic framework for interpretable affordance grounding, which identifies the objects in a scene that enable a given action (e.g., “cut”). CRAFT integrates structured commonsense priors from ConceptNet and language models with visual evidence from CLIP, using an energy-based reasoning loop to refine predictions iteratively. This process yields transparent, goal-driven decisions to ground symbolic and perceptual structures. Experiments in multi-object, label-free settings demonstrate that CRAFT enhances accuracy while improving interpretability, providing a step toward robust and trustworthy scene understanding.

Track: Neurosymbolic Methods for Trustworthy and Interpretable AI

Paper Type: Short Paper

Resubmission: No

Submission Number: 28