Anytime heuristic search algorithms try to find a (potentially suboptimal) solution as quickly as possible and then work to find better and better solutions until an optimal solution is obtained or time is exhausted. The most widely-known anytime search algorithms are based on best-first search.  In this paper, we propose a new algorithm, triangle search, that is based on beam search. Experimental results on a suite of popular search benchmarks suggest that it is competitive with fixed-width beam search and often performs better than the previous best anytime search algorithms.

Triangle Search: An Anytime Beam Search

Planning is the fundamental ability of an intelligent agent to reason about what decisions it should make in a given environment to achieve a particular set of goals. Generalized planning is the task of finding a generalized policy that applies to a set of planning instances that share a standard model. Action Schema Networks (ASNets) is an approach to find generalized policies for classical planning problems. In this paper, we extend ASNet to work with numeric planning problems. We use a technique to propositionalize numeric variables, which converts them from infinite ranges to a finite domain, and update the training procedure to use exploration in order to increase the diversity of states encountered. We also use a non-generalized numeric planner, Expressive Numeric Heuristic Search Planner (ENHSP), to teach ASNet to solve numeric planning problems by learning to mimic the actions chosen by ENHSP for problem instances. ASNet finds a generalized policy and weights after training, allowing it to share these to solve unseen problem instances of the same domain. We analyze our approach through an extensive experimental study aimed at evaluating the performance of ASNet on several numeric planning domains. The results show that our numeric ASNet can effectively solve problems in many numeric planning domains.

Action Schema Networks for Numeric Planning

The problem of navigation among moving obstacles, sometimes referred to as SIPP, is a problem in which the applicability of an action, such as moving to a particular neighboring location, can change with the passage of time. This means that the optimal plan, and its duration, can change depending on when execution begins. In practice, the execution start time is often unknown until planning completes or another agent gives the go-ahead. However, most prior work on SIPP assumes a known start time and thus that the found plans are always feasible.
In this paper, we relax this assumption and directly address the setting of any-start-time planning. We present a general-purpose data structure that compactly encodes the optimal plan as a function of start time, as well as planning algorithms that utilize it. In experiments on SIPP, we find that this data structure tells us how long our paths remain valid, while its overhead is minimal. We also find that, in any-start-time SIPP, replanning is insufficient for difficult problems. In contrast, our any-start-time algorithms can be quickly queried for the optimal plan once the start time is known, albeit at the cost of substantial precomputation time.

Any-Start-Time Planning for SIPP

This paper introduces an algebraic formalism linking transformation semigroups and the state transition systems induced by classical planning problems. We investigate some basic planning problems with interesting properties and establish fundamental characteristics of the corresponding semigroups, such as their ideals and Green's relations. Furthermore, we leverage semigroup theory to propose new approaches to existing concepts in automated planning, including the identification of landmark actions and the study of dead ends. We demonstrate that algebraic results can be applied to facilitate an understanding of a planning problem's state space and explore its solutions, thus verifying the relevance and effectiveness of such formal modeling.

From State Spaces to Semigroups: Leveraging Algebraic Formalism for Automated Planning

Combinatorial reconfiguration studies how one solution of a combinatorial problem can be transformed into another. The transformation can only make small local changes and may not leave the solution space. An important example is the independent set reconfiguration (ISR) problem, where an independent set of a graph (a subset of its vertices without edges between them) has to be transformed into another one by a sequence of modifications that remove a vertex or add another that is not adjacent to any vertex in the set. The 1st Combinatorial Reconfiguration Challenge (CoRe Challenge 2022) was a competition focused on the ISR problem. Our team participated with two solvers that model the ISR problem as a planning problem and employ different planning techniques for solving it. They successfully competed in the challenge and were awarded 4 first, 3 second, and 3 third places across 9 tracks. In this work, we present the ISR problem as a new problem to the planning community and describe the planning techniques used in our solvers. We re-ran the entire competition under equal computational conditions to allow for a fair comparison. Besides showcasing the success of planning technology, we hope that this work will create a cross-fertilization of the two research fields.

PARIS: Planning Algorithms for Reconfiguring Independent Sets

Balancing exploration and exploitation has been an important problem in both adversarial games and automated planning. While it has been extensively analyzed in the Multi-Armed Bandit (MAB) literature, and the game community has achieved great success with MAB-based Monte Carlo Tree Search (MCTS) methods, the symbolic planning community has struggled to advance in this area. We describe how Upper Confidence Bound 1’s (UCB1’s) assumption of reward distributions with known bounded support shared among siblings (arms) is violated when MCTS/Trial-based Heuristic Tree Search (THTS) in previous work uses heuristic values of search nodes in classical planning problems as rewards. To address this issue, we propose a new Gaussian bandit, UCB1-Normal2, and analyze its regret bound. It is variance-aware like UCB1-Normal and UCB-V, but has a distinct advantage: it neither shares UCB-V’s assumption of known bounded support nor relies on UCB1-Normal’s unfounded conjectures on Student’s t and χ2 distributions. Our theoretical analysis predicts that UCB1-Normal2 will perform well when the estimated variance is accurate, which can be expected in deterministic, discrete, finite state-space search, as in classical planning. Our empirical evaluation confirms that MCTS combined with UCB1-Normal2 outperforms Greedy Best First Search (traditional baseline) as well as MCTS with other bandits.

Scale-Adaptive Balancing of Exploration and Exploitation in Classical Planning

Planning tasks succinctly represent labeled transition systems, with each ground action corresponding to a label. This granularity, however, is not necessary for solving planning tasks and can be harmful, especially for model-free methods. In order to apply such methods, the label sets are often manually reduced.  In this work, we propose automating this manual process. We characterize a valid label reduction for classical planning tasks and propose an automated way of obtaining such valid reductions by leveraging lifted mutex groups. Our experiments show a significant reduction in the action label space size across a wide collection of planning domains.  We demonstrate the benefit of our automated label reduction in two separate use cases: improved sample complexity of model-free reinforcement learning algorithms and speeding up successor generation in lifted planning. The code and supplementary material are available at https://github.com/IBM/Parameter-Seed-Set.

On Reducing Action Labels in Planning Domains

Partial order reduction techniques are successfully used for various settings in planning, such as classical planning with A* search or with decoupled search, fully-observable nondeterministic planning with LAO*, planning with resources, or even goal recognition design. Here, we continue this trend and show that partial order reduction can be used for top-quality planning with K* search. We discuss the possible pitfalls of using stubborn sets for top-quality planning and the guarantees provided. We perform an empirical evaluation that shows the proposed approach to significantly improve over the current state of the art in unordered top-quality planning.

K* and Partial Order Reduction for Top-quality Planning

Top-k planning is a key formalism for many planning applications. K* search is a well-established approach to top-k planning. The algorithm iteratively runs A* search and Eppstein’s algorithm until a sufficient number of plans is found. The performance of K* algorithm is therefore inherently limited by the performance of A*, and in order to improve K* performance, that of A* must be improved. In cost-optimal planning, orbit space search improves A* performance, essentially performing A* in the orbit space instead of state space. In this work, we take a similar approach to top-k planning. We show theoretical equivalence between the goal paths in the state space and in the orbit space, allowing to perform K* search in the orbit space instead, reconstructing plans from the found paths in the orbit space. We prove that our algorithm is sound and complete for top-k planning and empirically show it to achieve state-of-the-art performance, overtaking all existing to date top-k planners.

K* Search Over Orbit Space for Top-k Planning

Finding multiple high-quality plans is essential in many planning applications, and top-k planning asks for finding k best plans, naturally extending cost-optimal classical planning. Several attempts have been made to formulate top-k classical planning as k-shortest paths finding problem and apply K* search, which alternates A* and Eppstein’s algorithm. However, earlier work had shortcomings, among which were failing to handle inconsistent heuristics and degraded performance in Eppstein’s algorithm implementations. As a result, existing evaluation results severely underrate the performance of K* based approach to top-k planning. In this paper, we present a new top-k planner based on a novel variant of K* search. We address the following three aspects. First, we show an alternative implementation of Eppstein’s algorithm for classical planning, which resolves a major bottleneck in earlier attempts. Second, we present a new strategy for alternating A* and Eppstein’s algorithm, which improves the performance of K* on classical planning benchmark. Last, we introduce a simple mitigation of the limitation of K* to tasks with a single goal state, allowing us to preserve heuristic informativeness in face of imposed task reformulation. Empirical evaluation results show that the proposed approach achieves the state-of-the-art performance on the classical planning benchmark.

On K* Search for Top-k Planning

It has been shown recently that successful techniques in classical planning, such as goal-oriented heuristics and landmarks, can improve the ability to compute planning programs for generalized planning (GP) problems. In this work, we introduce the notion of action novelty rank, which computes novelty with respect to a planning program, and propose novelty-based generalized planning solvers, which prune a newly generated planning program if its most frequent action repetition is greater than a given bound $v$, implemented by novelty-based best-first search BFS($v$) and its progressive variant PGP($v$). Besides, we introduce lifted helpful actions in GP derived from action schemes, and propose new evaluation functions and structural program restrictions to scale up the search. Our experiments show that the new algorithms BFS($v$) and PGP($v$) outperform the state-of-the-art in GP over the standard generalized planning benchmarks. Practical findings on the above-mentioned methods in generalized planning are briefly discussed.

Novelty and Lifted Helpful Actions in Generalized Planning

Collaborative multi-agent privacy preserving planning (CPPP) models problems where agents must work together to achieve joint goals, while keeping some information private. Recently, CPPP was extended to the stochastic case, where actions may fail, producing different effects than intended. Stochastic CPPP (SCPPP) problems can be solved using offline algorithms, such as RTDP. However, in many cases, we are not interested in computing a complete policy offline, and prefer to use an online approach, where one decides online on the next action only, without exploring the complete state space. This can allow us to scale to much larger problems. In this paper we thus explore online approaches for SCPPP. We suggest using a variant of the well known FF-Replan approach, adapted to CPPP, and a plan repair approach, where we try to locally return to the plan if an undesirable effect has occurred. We provide an empirical evaluation, comparing our approaches to an offline solver, showing that we can scale to much larger problems, and analyzing the strengths and weaknesses of our methods.

ICAPS 2023 Workshop HSDIP Submissions