Multi-Modal Attention Framework for Underwater Bioacoustic Denoising and Recognition

Amine Razig, Soulaymani Youssef, Loubna Benabbou, Pierre Cauchy

Track: Track 1: Original Research/Position/Education/Attention Track

Keywords: Learning from acoustics, Computer Vision, Biodiversity, Oceans and Marine Systems.

TL;DR: We propose a segmentation-driven, attention-guided framework that fuses masks with spectrograms to improve marine mammal call classification in noisy recordings, enabling robust biodiversity monitoring.

Abstract: Automated monitoring of marine mammals in the St. Lawrence Estuary faces extreme challenges: calls span low-frequency moans to ultrasonic clicks, often overlap, and are embedded in variable anthropogenic and environmental noise. We introduce a multi-modal, attention-guided framework that first segments spectrograms to generate soft masks of biologically relevant energy and then fuses these masks with the raw inputs for multi-band, denoised classification. Image and mask embeddings are integrated via mid-level fusion, enabling the model to focus on salient spectrogram regions while preserving global context. Using real-world recordings from the Saguenay–St. Lawrence Marine Park Research Station in Canada, we demonstrate that segmentation-driven attention and mid-level fusion improve signal discrimination, reduce false positive detections, and produce reliable representations for operational marine mammal monitoring across diverse environmental conditions and signal-to-noise ratios. By integrating attention-guided denoising with biodiversity-oriented evaluation metrics, our framework transforms raw hydrophone data streams into robust, operationally actionable presence signals, thereby supporting marine biodiversity conservation and climate-adaptation monitoring initiatives.

Submission Number: 400