
## Learning and Exploiting Progress States in Greedy Best-First Search
Patrick Ferber, Liat Cohen, Jendrik Seipp, Thomas Keller
Keywords: Search: Heuristic Search, Planning and Scheduling: Learning in Planning and Scheduling, Planning and Scheduling: Search in Planning and Scheduling, Search: Search and Machine Learning
IJCAI/2022/Proceedings/0657 - Learning and Exploiting Progress States in Greedy Best-First Search.pdf

### Implementation
_Given the documentation given by the authors on the method, how much time investment would it be to re-implement the method from scratch?_

[1]

The authors make a few statements regarding the implementation in 5.1. such as libraries used and state all the data is available online , for which they provide a citation where a zenodo link can be found (https://zenodo.org/records/6496716). In the zip file they present their code, benchmarks including data sets, log files. In the readme they state how to build the code, generate the logic features description, learn the decision tree (with a seperate readme regarding this) and how to parametrise this. They also present the intermediate data from their experiments. The code has some comments, but due to the size of the repository more would have been welcome. However the implementation does have decent documentation. 

### Data
_Given the data description in the documentation, how much effort take to either: Find the same dataset the authors used, or similar datasets and defend the comparability, or acquire one from scratch?_

[2]

(8/8)

The authors state the benchmarks they use to evaluate their method on in 5.1. They provide statistics on each in table 1, and a citations on it in 5.1. The files of six datasets are available in the implementation, the other 2 are possibly also in there under a different name but would have to be verified. Descriptions are given for only 2.

### Configuration 
_Given the (hyper)parameters, including semantic parameters, of the method: How much effort would it take to acquire the algorithm configurations used for their results, and compare against their budgetary constraints?_

[3]

The authors state they define a parameter space for each domain but this is in regards to generating training/validation instances. The authors do state they train decision tree on their generated feature sets, but details ofthis are missing. It does seem the algorithm configuration matters much less on their method as a whole, and the values could be extracted with some effort from the implementation.

### Experimental Procedure
_Given the experimental set-up of the work, how difficult is it to set up a new experiment, similar to those presented in the original work, with the same procedure?_

[3]

The authors use F1 score as metric, as well as geometric mean, and other domain specific metrics. The results seem to be signle runs. It requires some domain knowledge to fully understand.

### Expertise
_How much effort would it take to acquire the expertise required to reproduce the work independently relying on the available documentation?_

[6]

Requires experience in feature extraction/generation and progress states. 
