Modeling Conceptual Understanding in Image Reference Games

Rodolfo Corona Rodriguez, Zeynep Akata, Stephan Alaniz

Abstract: An AI system interacting with a wide population of other agents needs to be aware that there may be variations in the understanding that other agents have of the environment. Furthermore, not only can there be variation in agents' understanding, the machinery which they use to perceive the world may be inherently different, as is the case between humans and machines. In this work, we propose an image reference game played between a speaker and a population of listeners as an example of a setting where reasoning about which concepts other agents can comprehend is necessary. Our experiments on three benchmark image/attribute datasets indeed suggest that our learner encodes information directly pertaining to the understanding of other agents.

Code Link: https://github.com/rcorona/conceptual_img_ref

CMT Num: 7210