Cognition-Supervised Learning: Contrasting EEG Signals and Visual Stimuli For Saliency Detection
Primary Area: applications to neuroscience & cognitive science
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Electroencephalography, EEG, Contrastive Learning, Generative Modeling, Neuroimaging
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: In the rapidly evolving landscape of machine learning, the quest for efficient and accurate supervision signals remains paramount. Suitable supervision signals can be costly and practically impossible to obtain for models that require subjective cognitive labels, such as individual-specific interpretation of images or subjective training input for generative models. In this paper, we introduce a novel paradigm: cognition-supervised learning, leveraging human brain signals as direct supervisory signals. Using electroencephalogram (EEG) data, we contrastively train models to detect visual saliency without the need for any manual annotations. Our approach, the first of its kind, demonstrates that representations of semantic visual saliencies can be learned directly from EEG data. In downstream tasks, such as classification, clustering, and image generation, our learned representations not only reflect semantic saliency but also achieve competitive performance compared to models trained with manually labeled datasets. This work provides a promising avenue for future research in utilizing signals measured from the human cognitive system for supervising computer vision and machine learning models.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1665
Loading