Comparing Three Data Representations for Music with a Sequence-to-Sequence Model

Sichao Li, Charles Patrick Martin

Abstract: The choices of neural network model and data representation, a mapping between musical notation and input signals for a neural network, have emerged as a major challenge in creating convincing models for melody generation. Music generation can inspire creativity in artists and the general public, but choosing a proper data representation is complicated because the same musical piece can be presented in a range of expressive ways. In this paper, we compare three different data representations on the task of generating melodies with a sequence-to-sequence model, which generates melodies with flexible length, to explore how they affect the performance of generated music. These three representations are: a monophonic representation, playing one note each time, a polyphonic representation, indicating simultaneous notes and a complex polyphonic representation, expanding the polyphonic representation with dynamics. The influences of three data representations on the generated performance are compared and evaluated by mathematical analysis and human-cantered evaluation. The results show that different data representations fed into the same model endow the generated music with various features, the monophonic representation makes the music sound more melodious to humans’ ears, the polyphonic representation provides expressiveness and the complex-polyphonic representation guarantees the complexity of the generated music.