Energy Calibration Head: A Plug-In Neural Network Head with Human-like Uncertainty

Myoung Hoon Ha, Jungwon Ryu, ChangHwa Lee, SungHeon Jeong, Geon Yeong Park, Sang Wan Lee

Keywords: Metacognition, Overconfidence, Uncertainty, Energy, OOD

TL;DR: We propose a novel energy calibration head to alleviate overconfidence in OOD samples with a simple structure.

Abstract: The ability to distinguish what it knows from what it does not, known as metacognition, has been one of the fundamental challenges in modern AI. One benefit of metacognition is that it could preclude overconfident learning about out-of-distributions. For instance, machine learning models often exhibit excessive confidence when dealing with uncertain inputs. To mitigate this issue, we leverage the relationship between the marginal probability and conditional uncertainty found in our human behavioral experiments classifying out-of-distribution (OOD) images. Theoretical analyses reveal that uncertainty and marginal energy are loosely related and significantly influenced by the latent vector norm. Building upon this finding, we propose a novel plug-in type layer: energy calibration head (ECH). The ECH uses a metacognition module that calibrates uncertainty by evaluating the difference between actual marginal energy (indicative of how much it knows) and the marginal energy predicted based on the uncertainty level, leading to the attenuated joint energies for the OOD samples. We showed that a neural network with ECH emulates human-like uncertainty in OOD images (45.1% AUROC error reduction on average compared to a linear head) and can effectively perform anomaly detection tasks.

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3541