Artificial Dancing Intelligence: Neural Cellular Automata for Visual Performance of Music

Carlos Mariano Salcedo, Eran Egozy

Keywords: neural cellular automata, interactive visualizations, music visualizations, real-time systems, web-based applications, generative media

TL;DR: Web app for the visual performance of music powered by a novel audio-modulated Neural Cellular Automata architecture that runs on consumer devices.

Abstract:

We present Artificial Dancing Intelligence (ADI), an interactive neural music visualizer
that is accessed through a web app, but performs inference entirely on local devices. Our
approach enables anyone to create music-driven visuals while leveraging the expressive and
sometimes unpredictable dynamics of self-organized systems. ADI uses an audio stream’s
average energy (known as RMS) to modulate a neural cellular automata (NCA) that pro-
duces visual patterns that move and ’dance’ along with the audio stream in real-time.
Through the web interface, users can adjust the relationship between the music’s energy
and the NCA system to create unique visual performances out of any music audio stream.
ADI achieves smooth, real-time responsiveness on modern consumer devices.

Submission Number: 15