Variational Autoencoders for Polyphonic Music Interpolation

Pablo López Diéguez, Von-Wun Soo

Abstract: This paper aims to use machine learning techniques to solve the novel problem of music interpolation composition. Two models based on Variational Autoencoders (VAEs) are proposed to generate a suitable polyphonic harmonic bridge between two given songs, smoothly changing the pitches and dynamics of the interpolation. The interpolations generated by the first model surpass a Random data baseline and a bidirectional LSTM approach and its performance is comparable to the current state-of-the-art. The novel architecture of the second model outperforms the state-of-the-art interpolation approaches in terms of reconstruction loss by using an additional neural network for direct estimation of the interpolation encoded vector. Furthermore, the Hsinchu Interpolation MIDI Dataset was created, making both models proposed in this paper more efficient than previous approaches in the literature in terms of computational and time requirements during training. Finally, a subjective evaluation was done in order to ensure the validity of the metric-based results.