Groovy Pixels: Generating Drum Set Rhythms from Images

Yanjia Zhang

Abstract: It is a consensus that auditory and visual information can be quite similar in terms of the expression of emotions and knowledge. To explore this relationship with machine learning, this paper proposes a feasible system to generate drum beats from images. Specifically, the model converts the input image to an embedding vector, calculates a corresponding music embedding of a 4-bar drum set performance for this image embedding, and converts it to a playable MIDI file. The training process of the model is implemented by categorising the source dataset into the same set of genres and training with different combinations of images and drum beat for each genre. This paper also includes an evaluation of the performance of the system under different configurations.