ErrorAug: Making Errors to Find Errors in Semantic Segmentation

Devin Guillory, Dequan Wang, Shun Ishizaka, Kazuki Kozuka, Trevor Darrell

Keywords: Semantic Segmentation, Uncertainty Quantification, Error Detection

Abstract:

In order to develop trustworthy downstream applications for semantic segmentation models, it is important to not only understand the performance of a model on datasets, but to localize areas where the model may produce errors.
Pixel-wise error prediction of semantic segmentation maps is a challenging problem in which prior work relies on complicated image resynthesis pipelines.
We introduce \it{error augmentation}, a framework which enables us to learn robust error detectors by applying data transformations independently on the predicted segmentation maps.
This approach enables direct prediction of pixel-wise error in semantic segmentation maps, an approach explored as a naive baseline in prior works, to achieve state of the art performance.
As a proof-of-concept we propose a series of three simple transformations that generate challenging segmentation errors by swapping pixel predictions within a segmentation map.
Our approach outperforms previous methods of error detection for semantic segmentation across all metrics and improves performance by over $7.8\%$ on AUPR-Error.
Additionally, we show that our approach not only generalizes to unseen test examples, but remains reliable despite significant shifts in the target domain.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning

TL;DR: Introduces Error Augmentation as a framework for reliably producing error detectors for semantic segmentation with less architectural and computational complexity.