Seconds-Aligned PCA-DAC Latent Diffusion for Symbolic-to-Audio Drum Rendering

Konstantinos Soiledis, Maximos Kaliakatsos-Papakostas, Dimos Makris, Konstantinos Theodoros Tsamis

Keywords: drum synthesis, symbolic-to-audio, latent diffusion, audio codec, PCA, music generation, neural audio synthesis, onset detection, waveform synthesis, music information retrieval

Domains: Machine Learning Theory

Abstract:

Symbolic-control drum generation requires preserving explicit event timing and dynamics
while synthesizing acoustically plausible waveforms. We present a conditional
latent-diffusion model for symbolic-to-audio drum rendering. The model conditions on event
features sampled in physical time at codec-frame locations and predicts standardized principalcomponent coordinates of frozen DAC summed-codebook embeddings rather than waveform
samples. In the evaluated DAC configuration, 72 principal components capture the observed
training-frame summed-latent subspace under the stated SVD threshold, yielding a compact
continuous denoising target with a deterministic reconstruction path to the 1024-dimensional
DAC latent space before waveform decoding.
Across 1,733 held-out four-beat windows, PCA diffusion improves paired spectral and transient
metrics over deterministic PCA regression and a symbolic rendering baseline, while direct
regression remains stronger on phase-sensitive waveform L1. Auxiliary RVQ cross-entropy
improves short-step diffusion on mel error, onset-flux cosine, and waveform L1, with the most
favorable trade-offs occurring at 6–25 denoising steps depending on the metric.

Submission Number: 142