Insights from the 2018 IPC Benchmarks

Anonymous

Keywords: Automated Planning, International Planning Competition, Benchmarks Analysis

Abstract:

The International Planning Competition (IPC) empirically evaluates state-of-the-art planning systems on a set of benchmark problems. The selection of this benchmarks plays an important role in the competition, since they can significantly affect competition results.

In this paper we analyze the diversity of the benchmarks employed in the last IPC through extracting some features from the domains and problems of the optimal track. Finally, we provide some insights from the collected data and propose to use a similar method to select the benchmarks of future competitions.

Track: long (30 minutes)