Articulated Object Estimation in the Wild

RSS 2025 Workshop EgoAct Submission10 Authors

Keywords: Articulated Object Estimation, 3D Scene Understanding, Interactive Perception

TL;DR: We propose a novel method that estimates the motion model of articulated objects using point trajectories under partial or imperfect views of target objects given ego-centric RGB-D videos that we publish as the Arti4D dataset.

Abstract: Understanding the 3D motion of articulated objects is essential in robotic scene understanding, mobile manipulation, and motion planning. Prior methods for articulation estimation have primarily focused on controlled settings, assuming either fixed camera viewpoints or direct observations of various object states, which tend to fail in more realistic unconstrained environments. In contrast, humans effortlessly infer articulation by watching others manipulate objects. Inspired by this, we introduce ArtiPoint, a novel estimation framework that can infer articulated object models under dynamic camera motion and partial observability. By combining deep point tracking with a factor graph optimization framework, ArtiPoint robustly estimates articulated part trajectories and articulation axes directly from raw RGB-D videos. To foster future research in this domain, we introduce Arti4D, the first ego-centric in-the-wild dataset that captures articulated object interactions at a scene level, accompanied by articulation labels and ground-truth camera poses. We benchmark ArtiPoint against a range of classical and modern deep learning baselines, demonstrating its superior performance on Arti4D. We make code and Arti4D publicly available at redacted for anonymous review.

Submission Number: 10