Whisper-SV: Adapting Whisper for low-data-resource speaker verification

Li Zhang, Ning Jiang, Qing Wang, Yue Li, Quan Lu, Lei Xie

Abstract: Highlights•Whisper-SV adapts Whisper for speaker verification (SV) tasks.•The Representation selection module selects the top-k layers with most speaker info.•The multi-layer aggregation module merges top-k layers into a compact form for SV.•Whisper-SV features parameters-efficient and low training data requirements.•Experiments show Whisper-SV outperforms other methods in low-data-resource SV tasks.