What Matters in Hierarchical Search for Solving Combinatorial Problems?

TMLR Paper4945 Authors

Abstract:

Combinatorial problems, particularly the notorious NP-hard tasks, remain a significant challenge for AI research. A common approach to addressing them combines search with heuristics learned from demonstrations. Recently, hierarchical planning has emerged as a powerful framework in this context, enabling agents to decompose complex problems into manageable subgoals. However, the foundations of this approach, particularly the behavior and limitations of learned heuristics, remain underexplored. Our goal is to advance research in this area and establish a solid conceptual and empirical foundation.
Specifically, we identify the following key characteristics, whose presence favors the choice of hierarchical search methods: hard-to-learn value functions, complex action spaces, presence of dead ends in the environment, or training data collected from diverse sources. Through in-depth empirical analysis, we establish that hierarchical search methods consistently outperform standard search methods across these dimensions, and we formulate insights for future research. On the practical side, we also propose a set of evaluation guidelines to enable meaningful comparisons between methods and reassess the state-of-the-art algorithms.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=qu6D7zsIea

Changes Since Last Submission:

We thank the reviewers for their thoughtful feedback and encouragement to resubmit. We have carefully addressed all the concerns raised in the comments.

**Summary of Changes:**
1. **Scope & Framing:**
- We clarified throughout the manuscript that our results and analysis focus specifically on *subgoal-based hierarchical search methods with learned heuristics*.
- Section 4 and related results are now clearly introduced as properties of subgoal-based methods, and we *explicitly frame subgoal search* as a minimal instance of hierarchical search.
2. **Assumptions & Data Requirements:**
- The abstract, introduction, and contributions now state clearly that our approach relies on heuristics *learned from demonstrations*, which need not be optimal (e.g., reversed random walks are often sufficient).
- We *clarify the role of imitation learning* and provide context on the nature of the required data.
3. **Clarity in Contributions:**
- All claims and contributions are now **precisely stated** as applying under the assumption of *learned heuristics trained via imitation learning*, and we have avoided any implication of broader generalization than what is directly supported by our results.
4. **Wall-Clock Time and Search Budget:**
- We now provide **wall-clock time comparisons** and discuss the limitations of wall-time metrics in both the main text and appendix.

Assigned Action Editor: ~Marc_Lanctot1

Submission Number: 4945