UniFField: A Generalizable Unified Neural Feature Field for Visual, Semantic, and Spatial Uncertainties in Any Scene

Christian Maurer, Snehal Jauhri, Sophie Lueth, Georgia Chalvatzaki

Keywords: active perception, multi-modal, foundation models, uncertainty-aware, exploration, 3D scene understanding, feature field

Abstract: Comprehensive visual, geometric, and semantic understanding of a 3D scene is crucial for the successful execution of robotic tasks, especially in unstructured and complex environments. While recent 3D neural feature fields enable robots to leverage pretrained vision models for tasks such as language-guided manipulation and navigation, existing methods are typically scene-specific and do not model prediction uncertainty. We present UniFField, a unified uncertainty-aware neural feature field that combines visual, semantic, and geometric features in a single generalizable representation while also predicting uncertainty in each modality. Our approach generalizes zero-shot to any new environment, incrementally integrates RGB-D images into our voxel-based feature representation as the robot explores the scene, simultaneously updating uncertainty estimation. We evaluate the quality of the uncertainty predictions and demonstrate their effectiveness in an active object search task with a mobile manipulator robot.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 23