In-Context Learning for Pure Exploration

Alessio Russo, Ryan Welch, Aldo Pacchiano

Track: Theory

Keywords: pure exploration, active testing, reinforcement learning

Abstract:

In this work, we study the active sequential hypothesis testing problem, also known as pure exploration, where the goal is to actively control a data collection process to efficiently identify the correct hypothesis underlying a decision problem.
While relevant across multiple domains, devising adaptive exploration strategies remains challenging, particularly due to difficulties in encoding appropriate inductive biases. To address these limitations, we introduce In-Context Pure Exploration (ICPE), an in-context learning approach that uses Transformers to learn exploration strategies directly from experience. Numerical results across diverse benchmarks highlight ICPE's capability to achieve satisfactory performance in stochastic and structured settings, demonstrating its ability to meta-learn exploration strategies.

Serve As Reviewer: ~Ryan_Welch1, ~Alessio_Russo1

Submission Number: 38