GenParticles: Probabilistic Particle-Based Modeling for Object-Centric Motion

Arijit Dasgupta, Eric Li, Mathieu Huot, William T. Freeman, Vikash Mansinghka, Joshua B. Tenenbaum

Keywords: Bayesian Inference, Computer Vision, Markov-Chain Monte Carlo, Scene Understanding

TL;DR: GenParticles is a probabilistic particle-based model that infers object structure and motion from RGB video

Abstract: Modeling and manipulating the physical world from visual input requires tracking entities and inferring their structure under uncertainty. We introduce GenParticles, a probabilistic, particle-based generative model that that supports Bayesian inference of persistent object-level structure from observed positions and motion cues over time. The model defines latent particles representing spatially localized matter via 3D Gaussians and imposes hierarchical motion constraints by clustering particles into groups with coherent dynamics. Approximate inference is performed via parallelized block Gibbs sampling, facilitating tracking and refinement of latent structure across naturalistic video sequences with dense per-frame observations. GenParticles maintains temporally consistent inference by updating particle structure, allowing it to track both rigid and deformable motion without requiring explicit point correspondences. Beyond video analysis, this method offers an online framework for identifying and mapping moving objects within a scene, with potential relevance for downstream applications in robotic manipulation.

Submission Number: 6