Learning a metacognition for object detection

Marlene Berke, Mario Belledonne, Zhangir Azerbayev, Julian Jara-Ettinger

Keywords: metacognition, object detection, unsupervised learning

Abstract: In contrast to object recognition models, humans do not blindly trust their perception when building representations of the world, instead recruiting metacognition to detect percepts that are unreliable or false, such as when we realize that we mistook one object for another. We propose METAGEN, an unsupervised model that enhances object recognition models through a metacognition. Given noisy output from an object-detection model, METAGEN learns a meta-representation of how its perceptual system works and uses it to infer the objects in the world responsible for the detections. METAGEN achieves this by conditioning its inference on basic principles of objects that even human infants understand (known as Spelke principles: object permanence, cohesion, and spatiotemporal continuity). We test METAGEN on a variety of state-of-the-art object detection neural networks. We find that METAGEN quickly learns an accurate metacognitive representation of the neural network, and that this improves detection accuracy by filling in objects that the detection model missed and removing hallucinated objects. This approach enables generalization to out-of-sample data and outperforms comparison models that lack a metacognition.

One-sentence Summary: We present MetaGen, a model for the unsupervised learning of a metacognition for object detection systems that enhances accuracy and generalizability.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2110.03105/code)