ArtBeat - Deep Convolutional Networks for Emotional Inference to Enhance Art with Music

Liam Hebert, Elizabeth Eddy, Will Harrington, Lauryn Marchand, Jason d'Eon, Sageev Oore

Abstract: Paintings and music are two universal forms of art that are present across all cultures and times in human history. In this paper, we present ArtBeat, a machine learning application to connect the two. Not only are these two art forms universal, but they are also deeply emotionally charged. This emotional factor is what we use as a bridge between the mediums. Using a Convolutional Neural Network (CNN), we aimed to create a model that can classify the emotions evoked by a painting, and use the predicted values to pair it with a piece of music to complement the viewing experience. Our system uses a pre-trained Wide ResNet model as a base, which we then fine-tuned. In this paper, we describe the design and implementation of this model as well as report its results and analyze its behaviour.