SA-TTS: Stress-Aware Test-Time Scaling for Vision Models

youla yang

Keywords: test-time scaling, adaptive inference, uncertainty estimation, test-time augmentation, dynamic compute allocation, robustness

TL;DR: We propose SA-TTS, a lightweight framework that allocates inference compute adaptively using stress signals to improve the accuracy–compute trade-off in vision models.

Abstract:

Test-time scaling has recently emerged as an effective paradigm for improving reasoning and prediction performance by allocating additional computation during inference. While this idea has shown remarkable success in large language models, its potential in computer vision remains underexplored.

We propose Stress-Aware Test-Time Scaling (SA-TTS), a lightweight framework that dynamically allocates inference computation based on an estimated stress score, which serves as a proxy for prediction difficulty or uncertainty. The proposed method combines inexpensive uncertainty signals—including predictive entropy, margin ambiguity, and augmentation disagreement—to guide adaptive selection of inference policies such as single-pass inference, test-time augmentation, or multi-crop evaluation.

Across image classification benchmarks and corruption robustness settings, SA-TTS improves the accuracy–compute trade-off compared to fixed inference policies while maintaining competitive calibration behavior. Our approach is model-agnostic, requires no retraining of the backbone network, and can be integrated into existing vision pipelines with minimal overhead.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 7