Robust Neural Audio Fingerprinting using Music Foundation Models

Shubhr Singh, Kiran Bhat, Benjamin Resnick, Xavier Riley, John Thickstun, Walter De Brouwer

Keywords: Robust neural fingerprinting, Foundation model fingerprints, Data augmentation, Music identification, Segment-level localization

TL;DR: Music foundation models yield more accurate and robust audio fingerprints than existing methods

Abstract:

The proliferation of distorted, compressed, and manipulated music on modern media platforms like TikTok motivates the development of more robust audio fingerprinting techniques to identify the sources of musical recordings.
In this paper, we develop and evaluate new neural audio fingerprinting techniques with the aim of improving their robustness. We make two contributions to neural fingerprinting methodology: (1) we use a pretrained music foundation model as the backbone of the neural architecture and (2) we expand the use of data augmentation to train fingerprinting models under a wide variety of audio manipulations, including tempo changes, pitch modulation, compression, and filtering.
We systematically evaluate our methods in comparison to two state-of-the-art neural fingerprinting models: NAFP and GraFPrint. Results show that fingerprints extracted with music foundation models (e.g., MuQ, MERT) consistently outperform models trained from scratch or pre-trained on non-musical audio. Segment-level evaluation further reveals their capability to accurately localize fingerprint matches, an important practical feature for catalog management.

Track: Paper Track

Confirmation: Paper Track: I confirm that I have followed the formatting guideline and anonymized my submission.

Submission Number: 74