Towards Leveraging Sequential Structure in Animal Vocalizations

Eklavya Sarkar, Mathew Magimai Doss

Keywords: Bioacoustics, discretization, token sequences, quantization

TL;DR: Discretization of animal vocalizations into token sequences can preserve subunit-level sequential information and classify animal calls.

Abstract: Animal vocalizations contain sequential structures that carry important communicative information, yet most computational bioacoustics studies average the extracted frame-level features across the temporal axis, discarding the order of the sub-units within a vocalization. This paper investigates whether discrete acoustic token sequences, derived through vector quantization and gumbel-softmax vector quantization of extracted self-supervised speech model embeddings, can effectively capture and leverage temporal information. Pairiwse distance analysis of token sequences shows that they can discriminate call-types and callers across four bioacoustics datasets. Sequence classification experiments using $k$-Nearest Neighbour with Levenshtein distance show that vector-quantized token sequences yield reasonable call-type and caller classification performances, and hold promise as alternative feature representations towards leveraging sequential information in animal vocalizations.

Submission Number: 43