FOCUS: High-Dimensional Model-Based RL via Focused Control Unit Sampling

ICLR 2026 Conference Submission12925 Authors

Keywords: High-dimensional continuous control, Model-based reinforcement learning

Abstract: Reinforcement learning (RL) in high-dimensional continuous state and action spaces often struggles with low learning efficiency and limited exploration scalability. To address this, we introduce FOCUS, a novel model-based RL framework that leverages the insight that effective policies often rely on dynamically focused, sparse control. FOCUS learns preferences over action dimensions to facilitate more targeted and efficient policy learning. It employs a hierarchical decision-making strategy, in which a high-level policy generates binary prompts to activate control units that have more impact on the task performance, while a low-level policy produces actions conditioned on these prompts. To promote behavioral diversity guided by different control-unit preferences, we integrate a diversity-driven objective into the model-based policy optimization process. FOCUS significantly outperforms existing methods on multiple visual control tasks. Furthermore, it facilitates the integration of prior knowledge about the importance of action dimensions, making it particularly effective for complex, high-dimensional tasks.

Primary Area: reinforcement learning

Submission Number: 12925