On the miscalibration of object 6D pose estimation methods

Matteo Bortolon, Fabio Poiesi

Keywords: 6D Object Pose Estimation, Uncertainty Estimation, Confidence Calibration, Industrial Robotics, Robotic Errors

TL;DR: Confidence scores in 6D pose estimation are poorly calibrated, making them unsafe for industrial robots. Current calibration methods fail, highlighting the need for new, model-level solutions to ensure reliability.

Abstract:

Uncertainty estimation in 6D object pose estimation is crucial for industrial robotic applications, where short cycle times and near-optimal availability are essential. The primary concern is not just the accuracy of the pose, but the reliability of the confidence scores to prevent costly robotic errors, such as crashes.
Despite the great strides in accuracy demonstrated by leading methods in benchmarks like the BOP Challenge, we have observed that their confidence scores are often poorly calibrated.
This poor calibration results in a significant gap between a method's reported performance and its true reliability in industrial settings.
In this paper, we conduct a experimental study on this issue across multiple BOP datasets.
We demonstrate that even after applying state-of-the-art calibration techniques, this miscalibration persists.
Our findings highlight the limitations of current post-hoc calibration methods, which we found do not work ``out of the box" and have notable drawbacks.
This indicates a deeper, systemic issue that requires fundamental changes at the model or foundation model training level.
Our work underscores the critical need for new calibration approaches tailored to the unique challenges of 6D pose estimation for robust industrial deployment.

Submission Number: 5