Avi: A 3D Vision-Language Action Model Architecture generating Action from Volumetric Inference

Harris Song, Long Le

Keywords: Robotics, MLLM, VLA

TL;DR: Predicting Point Cloud Transformations for a World-Based Geometry Model instead of the status quo Robot-Based Action Model.

Abstract: We propose Avi, a novel 3D Vision-Language-Action (VLA) architecture that reframes robotic action generation as a problem of 3D perception and spatial reasoning, rather than low-level policy learning. While existing VLA models primarily operate on 2D visual inputs and are trained end-to-end on task-specific action policies, Avi leverages 3D point clouds and language-grounded scene understanding to compute actions through classical geometric transformations. Most notably, Avi does not train on previous action tokens, rather, we build upon a 3D Multi-modal Large Language Model (MLLM) to generate the next point cloud and explicitly calculate the actions through classical transformations. This approach enables generalizable behaviors that are robust to occlusions, camera pose variations, and changes in viewpoint. By treating the robotic decision-making process as a structured reasoning task over 3D representations, Avi bridges the gap between high-level language instructions and low-level actuation without requiring opaque policy learning. Our preliminary results highlight the potential of 3D vision-language reasoning as a foundation for scalable, robust robotic systems. Check it out at action-volume-inference.github.io.

Submission Type: Research Paper (4-9 Pages)

Submission Number: 92