Shared Visual Representations of Drawing for Communication: How do different biases affect human interpretability and intent?Download PDF

Published: 02 Nov 2021, Last Modified: 05 May 2023SVRHM 2021 PosterReaders: Everyone
Keywords: visual representations, representational losses, sketching agents, human perception
TL;DR: This paper shows that interpretable drawings can be generated by self-supervised artificial agents, playing a visual communication game, with appropriate inductive biases.
Abstract: We present an investigation into how representational losses can affect the drawings produced by artificial agents playing a communication game. Building upon recent advances, we show that a combination of powerful pretrained encoder networks, with appropriate inductive biases, can lead to agents that draw recognisable sketches, whilst still communicating well. Further, we start to develop an approach to help automatically analyse the semantic content being conveyed by a sketch and demonstrate that current approaches to inducing perceptual biases lead to a notion of objectness being a key feature despite the agent training being self-supervised.
10 Replies