DyPolySeg: Taylor Series-Inspired Dynamic Polynomial Fitting Network for Few-shot Point Cloud Semantic Segmentation

Changshuo Wang, Xiang Fang, Prayag Tiwari

Abstract: Few-shot point cloud semantic segmentation effectively addresses data scarcity by identifying unlabeled query samples through semantic prototypes generated from a small set of labeled support samples. However, pre-training-based methods suffer from domain shifts and increased training time. Additionally, existing methods using DGCNN as the backbone have limited geometric structure modeling capabilities and struggle to bridge the categorical information gap between query and support sets. To address these challenges, we propose DyPolySeg, a pre-training-free Dynamic Polynomial fitting network for few-shot point cloud semantic segmentation. Specifically, we design a unified Dynamic Polynomial Convolution (DyPolyConv) that extracts flat and detailed features of local geometry through Low-order Convolution (LoConv) and Dynamic High-order Convolution (DyHoConv), complemented by Mamba Block for capturing global context information. Furthermore, we propose a lightweight Prototype Completion Module (PCM) that reduces structural differences through self-enhancement and interactive enhancement between query and support sets. Experiments demonstrate that DyPolySeg achieves state-of-the-art performance on S3DIS and ScanNet datasets.

Lay Summary:

Imagine teaching a computer to recognize different objects in 3D scenes - like distinguishing chairs from tables - using only a few examples of each. This challenge, called few-shot learning, is crucial for robots and autonomous systems that encounter new environments with limited training data.

Current methods require extensive pre-training on large datasets, which is time-consuming and often fails when new data differs from training examples. It's like trying to recognize modern office furniture after only seeing traditional home furniture.

We developed DyPolySeg, a system that learns to identify 3D objects without extensive pre-training. Our approach has two key innovations: a flexible shape-analysis tool that automatically adapts to capture both simple flat surfaces and complex curved structures, and an intelligent comparison system that bridges differences between training examples and new objects to be identified.

Our experiments show DyPolySeg significantly outperforms existing methods, achieving better accuracy while being more efficient. This advancement could improve robotics, autonomous vehicles, and augmented reality applications where machines must quickly understand 3D environments with minimal training data.

Primary Area: Applications->Computer Vision

Keywords: Dynamic Convolution, Taylor Series, Prototype Completion, Few-shot, Point Cloud, Semantic Segmentation

Submission Number: 5054