Beyond Entropy: Style Transfer Guided Single Image Continual Test-Time Adaptation

Younggeol Cho, Youngrae Kim, Dongman Lee

Abstract:

Continual test-time adaptation methods are designed to facilitate the continual adaptation of models to dynamically changing real-world environments.
Concurrently, real-world vision applications, such as semantic segmentation, necessitate the utilization of high-resolution images to achieve optimal performance, which limits the batch size during test time.
However, the instability caused by batch normalization layers and entropy loss when using small batch sizes as well as a single image significantly destabilizes many existing methods in real-world continual TTA scenarios.
To overcome these challenges, we present BESTTA, a novel single image continual test-time adaptation method guided by style transfer, which enables stable and efficient adaptation to the target environment by transferring the style of the input image to the source style.
To implement the proposed method, we devise BeIN, a simple yet powerful normalization method, along with the style-guided losses.
We demonstrate that BESTTA effectively adapts to the continually changing target environment, leveraging only a single image on both semantic segmentation and image classification tasks.
Remarkably, despite training only two parameters in a BeIN layer consuming the least memory, BESTTA outperforms existing state-of-the-art methods in terms of performance.