Lipschitz-Guided Monte Carlo Tree Search with Knowledge Transfer across Sequential Tasks

ICLR 2026 Conference Submission14014 Authors

Keywords: Lipschitz, Lifelong Planning

Abstract: Monte Carlo Tree Search (MCTS) has proven highly effective in solving complex planning tasks by balancing exploration and exploitation using Upper Confidence Bound for Trees (UCT). However, existing works have not considered MCTS-based lifelong planning facing a sequence of MDPs -- e.g., each MDP with varying transition probabilities and rewards from previous ones -- throughout the operational lifetime. This paper presents LiZero for Lipschitz lifelong planning using MCTS. We propose a novel concept of adaptive UCT (aUCT) to transfer knowledge from previous tasks to the exploration/exploitation of a new task, depending on both the Lipschitz continuity between tasks and the confidence of knowledge in Monte Carlo action sampling. We analyze LiZero's acceleration factor in terms of improved sampling efficiency and also develop efficient algorithms to compute aUCT in an online fashion by both data-driven and model-based approaches, whose sampling complexity and error bounds are also characterized. Numerical results show that LiZero significantly outperforms existing MCTS and lifelong learning baselines in terms of much faster convergence (3$\sim$4x). Our results highlight the potential of LiZero to advance decision-making and planning in dynamic environments.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 14014