CatFree3D: Category-Agnostic 3D Object Detection With Diffusion

Wenjing Bian, Zirui Wang, Andrea Vedaldi

Keywords: Diffusion, 3D Object Detection, 3D Detection Metric

TL;DR: A state-of-the-art diffusion-based 3D object detection pipeline is introduced, enabling category-agnostic detection and enhancing accuracy, along with a new NHD metric for precise evaluation.

Abstract: Image-based 3D object detection is widely employed in applications such as autonomous vehicles and robotics, yet current systems struggle with generalisation due to the complex problem setup and limited training data. We introduce a novel pipeline that decouples 3D detection from 2D detection and depth prediction, using a diffusion-based approach to improve accuracy and support category-agnostic detection. Additionally, we introduce the Normalised Hungarian Distance (NHD) metric for an accurate evaluation of 3D detection results, addressing the limitations of traditional IoU and GIoU metrics. Experimental results demonstrate that our method achieves state-of-the-art accuracy and strong generalisation across various object categories and datasets.

Submission Number: 67