Toward Preference-Guided Interactive Steering of Real-Time Music Generation

Wonchul Shin, Kyogu Lee

Submission Type: 2-page Extended Abstract (Non-archival) / 2페이지 Extended Abstract (프로시딩 미수록)

Keywords: real-time music generation, interactive music generation, human-in-the-loop, preference learning

TL;DR: We present a prototype for steering real-time music generation through iterative user preference feedback, suggesting that musical output can be progressively guided toward preferred directions as it unfolds.

Abstract: Real-time music generation models open up new possibilities for continuous interaction, but steering such systems through text alone remains limited. Listeners often cannot precisely verbalize their musical preferences, and those preferences may evolve while listening. We explore a preference-guided interactive steering framework for real-time music generation, where a user listens to a continuously generated stream and provides preference feedback on successive audio segments. The system updates the style-conditioning signal online to support iterative exploration and steering toward user preferences. As a proof of concept, we implemented a prototype with a manually constructed low-dimensional control space and an online preference optimization loop. Early pilot sessions suggest that such steering is feasible when the control space provides consistent perceptual variation, while revealing challenges posed by noisy streaming interaction and the cost of human feedback for algorithm development. Our ongoing work focuses on constructing locally steerable control spaces, developing an embedding-based simulation environment for scalable algorithm development, and evaluating the resulting system through human studies.

Email Sharing: We authorize sharing author emails with Program Chairs. / 저자 이메일 공유에 동의합니다.

Data Release: We understand and agree to the OpenReview metadata visibility policy. / OpenReview 메타데이터 공개 정책을 이해하고 동의합니다.

Submission Number: 4