Planner Metrics Should Satisfy Independence of Irrelevant AlternativesDownload PDF


Apr 13, 2019 (edited Jul 12, 2019)ICAPS 2019 Workshop WIPC Blind SubmissionReaders: Everyone
  • Keywords: metrics
  • Abstract: We argue that planner evaluation metrics should satisfy the independence of irrelevant alternatives criterion, i.e., the decision whether planner A is ranked higher or lower than planner B should be independent of planner C. We show that three metrics used in classical planning competitions do not necessarily satisfy this criterion and highlight alternative metrics that do so.
  • Track: short (15 minutes)
2 Replies