Do Foundation Models Smell Like Humans?

Farzaneh Taleb, Miguel Serras Vasco, Nona Rajabi, Mårten Björkman, Danica Kragic

Track: short paper (up to 5 pages)

Keywords: olfactory perception, foundation models, alignment, similarity rating, Deep neural networks

TL;DR: In this paper we evaluate the alignement between representations encoded from foundation models trained on chemical structures and human olfactory perception.

Abstract: The human brain encodes stimuli from the environment into representations that form a sensory perception of the world. Despite recent advances in understanding visual and auditory perception, olfaction remains an under-explored topic in the machine learning community due to the lack of large-scale datasets annotated with labels related to human olfactory perception. Simultaneously, foundation models have recently demonstrated impressive performance in several tasks by leveraging large-scale datasets without a supervision signal. In this work, we ask the question of whether foundation models of chemical structures encode representations that are aligned with the human olfactory perception, i.e., do foundation models smell like humans? We demonstrate that representations encoded from foundation models pre-trained on general chemical structures are highly aligned with human olfactory perception.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 64