Solving Blind Non-linear Forward and Inverse Problem for Audio Applications

Jaehyun Park, Sungho Lee, Hayoon Kim, Jin Woo Lee, Dohoon Lee, Kyogu Lee

Keywords: audio effect, zero-shot system identification, inverse problem, diffusion model, sequential monte carlo

Abstract:

We propose a unified framework to address the blind forward and inverse problems in audio domain, where the objective is to estimate either the function or the input signal solely from the observed output, without access to the other.
We formally define forward operators ---mapping input to output signals --- and formulate both problems within a probabilistic framework.
For the blind forward problem, we design an architecture that utilizes a reference encoder to extract features from the reference signal, enabling the main operator to approximate arbitrary forward operators systematically composed via algebraic representations.
For the blind inverse problem, we employ a conditional diffusion model conditioned on features from the pretrained reference encoder and augment the generation process using twisted particle filtering technique leveraging the approximated operator in the forward problem.
We validate our framework on zero-shot audio effect modeling and speech enhancement. The experiments show that our approach replicates both simple and complex audio effects, generalizes under distribution mismatches, and effectively enhances noisy full-band audio across diverse effects and real-world scenarios.
Codes are available at https://t.ly/n11uk , with audio samples at https://t.ly/dBUhF

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9263