Embedding Alignment in Code Generation for Audio

Sam Kouteili, Hiren Madhu, George Typaldos, Mark Paul Santolucito

Keywords: Live Coding, Embedding Alignment, Code Generation, LLM

TL;DR: We build an embedding alignment map between live-coding code and generated audio output.

Abstract: Large Language Model (LLM) code generation has the potential to enhance creative coding by allowing users to focus on structural and musical motifs rather than syntactic details. For live-coding and other music-oriented settings, users would benefit from diverse candidates that reflect meaningful differences in the resulting audio. However, current models struggle to produce such diversity, as they lack direct insight into the code’s sonic output and are typically evaluated using text-based similarity metrics. In this paper, we propose a predictive MLP model that learns an embedding alignment map between code and audio, enabling reasoning about musical similarity directly from code embeddings. This alignment introduces musical awareness into code generation workflows, supporting more perceptually relevant candidate selection and opening the door to musically informed code assistants.

Track: Paper Track

Confirmation: Paper Track: I confirm that I have followed the formatting guideline and anonymized my submission.

Submission Number: 69