Artificial Dancing Intelligence: Neural Cellular Automata for Visual Performance of Music

Carlos Mariano Salcedo, Eran Egozy

Keywords: neural cellular automata, interactive visualizations, music visualizations, real-time systems, web-based applications, generative media

TL;DR: Web app for the visual performance of music powered by a novel audio-modulated Neural Cellular Automata architecture that runs on consumer devices.

Abstract:

We present Artificial Dancing Intelligence (ADI), an interactive neural music visualizer
that is accessed through a web app, but performs inference entirely on local devices. Our
approach enables anyone to visually express music, while leveraging the expressive, and
sometimes unpredictable dynamics of self-organized systems. ADI uses an audio stream’s
average energy (known as RMS) to modulate a neural cellular automata (NCA) to produce
visual patterns that move and ’dance’ along with the audio stream in real time. Through the
web interface, users can adjust the relationship between the music’s energy and the NCA
system to create unique visual performances out of any music audio file. ADI achieves
smooth, real-time responsiveness on modern consumer devices.

Submission Number: 15