CT3: Boosting Downstream Performance through Test-time Training on AI PCs with Remote Multi-Domain Knowledge Bases

ACL ARR 2025 May Submission881 Authors

Abstract: Learning at test time is an effective strategy for improving the performance of large language models (LLMs), although at the expense of increased computational costs during inference. In this paper, we introduce Collaborative Test-Time Training (CT3), a novel system designed to enhance the downstream accuracy of LLMs on client devices such as AI PCs through Test-Time Training (TTT) leveraging a remote multi-domain knowledge base. CT3 efficiently distributes the TTT process using a server-client architecture, allowing clients to fine-tune their models using relevant samples from the server. It also proposes a local state management mechanism and a simple but effective sample size reduction strategy to optimize test-time training without compromising accuracy. Our experiments demonstrate significant accuracy improvements across multiple domains and various LLMs with up to 44% increase in average downstream performance and a speedup ranging from 1.5× to 2.5× compared with vanilla TTT. The code to reproduce CT3’s results will be released open-source.

Paper Type: Long

Research Area: Information Retrieval and Text Mining

Research Area Keywords: Downstream NLP tasks, fine-tuning, information retrieval

Contribution Types: NLP engineering experiment, Approaches low compute settings-efficiency

Languages Studied: English

Submission Number: 881