The role of shared labels and experiences in representational alignment
Track: short paper (up to 5 pages)
Keywords: language, labels, vision, cognitive models
TL;DR: We test the role of shared labels and visual diet overlap in representational alignment between simulated neural network agents
Abstract: Successful communication is thought to require members of a speech community to learn common mappings between words and their referents. But if one person's concept of CAR is very different from another person's, successful communication might fail despite the common mappings because different people would mean different things by the same word. Here we investigate the possibility that one source of representational alignment is language itself. We report a series of neural network simulations investigating how representational alignment changes as a function of agents having more or less similar visual experiences (overlap in ``visual diet'') and how it changes with exposure to category names. We find that agents with more similar visual experiences have greater representational overlap. However, the presence of category labels not only increases representational overlap, but also greatly reduces the importance of having similar visual experiences. The results suggest that ensuring representational alignment may be one of language's evolved functions.
Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.
Submission Number: 43
Loading