Investigating Emotion-Color Association in Deep Neural Networks

Anonymous

Keywords: Deep Neural Network, Affective Computing, Emotional Intelligence, Emotion-Color Association

TL;DR: The representations learned by DNNs can indeed show an emotion-color association and can probably help us in the emotion classification task.

Abstract: Recent research has shown that Deep Neural Networks (DNNs) correlate very well to neural responses, and are widely used by cognitive scientists as a proxy for human representation to model human behavior. But previously it has not been explored whether DNNs capture any aspects of stimuli association. In this study, we experimentally investigate if DNNs can learn implicit associations in stimuli, particularly, an emotion-color association between image stimuli. Our study was conducted in two parts. First, we collected human responses on a forced-choice decision task in which subjects were asked to select a color for a specified emotion-inducing image. Next, we modeled this decision task on neural networks using the similarity between deep representation (extracted using DNNs trained on object classification tasks) of the stimuli images and images of colors used in the task. We found that our model showed a fuzzy linear relationship between the two decision probabilities. This results in two interesting findings, 1. The representations learned by deep neural networks can indeed show an emotion-color association 2. The emotion-color association is not just random but involves some cognitive phenomena. Finally, we also show that this method can help us in the emotion classification task.