The Ghost in the Keys: A Disklavier Demo for Human-AI Musical Co-Creativity

Louis Bradshaw, Alexander Spangher, Stella Biderman, Simon Colton

Keywords: music, piano, midi, disklavier, hci, creativity

TL;DR: We introduce a system for turn-based real-time human-ai co-creation on a Disklavier Paino

Abstract: While generative models for music composition are increasingly capable, their adoption by musicians is hindered by text-prompting, an asynchronous workflow disconnected from the embodied, responsive nature of instrumental performance. To address this, we introduce Aria-Duet, an interactive system facilitating a real-time musical duet between a human pianist and Aria, a state-of-the-art generative model, using a Yamaha Disklavier as a shared physical interface. The framework enables a turn-taking collaboration: the user performs, signals a handover, and the model generates a coherent continuation performed acoustically on the piano. After first describing the technical architecture enabling this low-latency interaction on consumer hardware, we analyze the resulting system through sessions with pianists, giving insights into the real-time auto-complete paradigm it presents. Overall, we find that this interactive framework works naturally with trained pianists, and that further post-training to the model could further improve the co-creative experience.

Track: Paper Track

Confirmation: Paper Track: I confirm that I have followed the formatting guideline and anonymized my submission.

(Optional) Short Video Recording Link: https://www.youtube.com/watch?v=8s3V922h3CU

Submission Number: 36