Coarse-to-Fine 3D Part Assembly via Semantic Super-Parts and Symmetry-Aware Pose Estimation

Xinyi Zhang, Bingyang Wei, Ruixuan Yu, Jian Sun

Keywords: 3D part assembly; semantic super-parts; dual-range feature propagation; symmetry-aware supervision

TL;DR: We propose CFPA for 3D part assembly that integrates semantic super-parts, dual-range feature propagation, and symmetry-aware supervision, achieving SOTA accuracy, consistency, and diversity on standard benchmarks.

Abstract: We propose a novel two-stage framework, Coarse-to-Fine Part Assembly (CFPA), for 3D shape assembly from basic parts. Effective part assembly demands precise local geometric reasoning for accurate component assembly, as well as global structural understanding to ensure semantic coherence and plausible configurations. CFPA addresses this challenge by integrating semantic abstraction and symmetry-aware reasoning into a unified pose prediction process. In the first stage, semantic super-parts are constructed via an optimal transport formulation to capture high-level object structure, which is then propagated to individual parts through a dual-range feature propagation mechanism. The second stage refines part poses via cross-stage feature interaction and instance-level geometric encoding, improving spatial precision and coherence. To enable diverse yet valid assemblies, we introduce a symmetry-aware loss that jointly models both self-symmetry and inter-part geometric similarity, allowing for diverse but structurally consistent assemblies. Extensive experiments on the PartNet benchmark demonstrate that CFPA achieves state-of-the-art performance in assembly accuracy, structural consistency, and diversity across multiple categories.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 385