Partial Rankings of Optimizers
Keywords: multicriteria benchmarking, ranking optimizers, partial order, ufg depth, benchmarking, optimizers, BBOB, deepOBS
TL;DR: We propose partially ranking optimizers according to multiple criteria in order to evaluate both benchmark suites and optimizers using a depth function for partial orders.
Abstract: We introduce a framework for benchmarking optimizers according to multiple criteria over a collection of test functions. Based on a recently introduced union-free generic depth function for partial orders/rankings, it fully exploits the ordinal information and allows for incomparability. Our method describes the distribution of all partial orders/rankings, avoiding the notorious shortcomings of aggregation. This permits to identify test functions that produce central or outlying rankings of optimizers and to assess the quality of benchmarking suites.
Submission Number: 116
Loading