Test-Time Training for Speech
Keywords: Speech, masked autoencoders, test-time training, self-supervised learning, distributional shift, emotion classification, speaker identification
TL;DR: In this paper, we extend test-time training (TTT) for speech and show that using parameter efficient techniques particularly, by fine-tuning just the bias terms during TTT, we can improve performance and efficiency.
Abstract: In this paper, we study the application of Test-Time Training (TTT) as a solution to handling distribution shifts in speech applications. In particular, we introduce distribution-shifts to the test datasets of standard speech-classification tasks---for example, speaker-identification and emotion-detection---and explore how Test-Time Training (TTT) can help adjust to the distribution-shift. In our experiments that include distribution shifts due to background noise and natural variations in speech such as gender and age, we identify some key-challenges with TTT including sensitivity to optimization hyperparameters (e.g., number of optimization steps and subset of parameters chosen for TTT) and scalability (e.g., as each example gets its own set of parameters, TTT is not scalable). Finally, we propose using BitFit -- a parameter-efficient fine-tuning algorithm proposed for text applications that only considers the bias parameters for fine-tuning -- as a solution to the aforementioned challenges and demonstrate that it is consistently more stable than fine-tuning all the parameters of the model.
Submission Number: 46
Loading