IGSPAD: Inverting 3D Gaussian Splatting for Pose-agnostic Anomaly Detection

Bolin Jiang, Yuqiu Xie, Jiawei Li, Naiqi Li, Bin Chen, Shu-Tao Xia

Abstract: Pose-agnostic anomaly detection refers to the situation where the pose of test samples is inconsistent with the training dataset, allowing anomalies to appear at any position in any pose. We propose a novel method IGSPAD to address this challenge. Specifically, we employ 3D Gaussian splatting to represent the normal information from the training dataset. To accurately determine the pose of the test sample, we introduce an approach termed Inverting 3D Gaussian Splatting (IGS) to address the challenge of 6D pose estimation for anomalous images. The pose derived from IGS is utilized to render a normal image well-aligned with the test sample. Subsequently, the image encoder of the Segment Anything Model is employed to identify discrepancies between the rendered image and the test sample, predicting the location of anomalies. Experimental results on the MAD dataset demonstrate that the proposed method significantly surpasses the existing state-of-the-art method in terms of precision (from 97.8% to 99.7% at pixel level and from 90.9% to 98.0% at image level) and efficiency.

Primary Subject Area: [Experience] Multimedia Applications

Relevance To Conference:

Our work contributes the following:
1) It resolves the 6D pose estimation for anomalous image problem using 3D Gaussian splatting;
2) It employs a 3D Gaussian model to address pose-agnostic anomaly detection issues in 2D images;
3) It substantially improves the accuracy and efficiency of pose-agnostic anomaly detection.

Submission Number: 5076