Solving Deceptive Problems Without Explicit Diversity Maintenance

Ryan Boldi, Li Ding, Lee Spector

Abstract: Navigating deceptive domains has often been a challenge in machine learning due to search algorithms getting stuck at sub-optimal local maxima. Many algorithms have been proposed to navigate these domains by explicitly maintaining population diversity, such as Novelty Search, MAP-Elites or other so-called Quality Diversity algorithms. In this paper, we present an approach to solve deceptive problems by optimizing a series of defined objectives instead. These objectives can be created by subaggregating the performance of individuals, and optimized by lexicase selection, a technique shown to implicitly maintain population diversity. We compare this method to a commonly used quality diversity algorithm, MAP-Elites, on a set of discrete optimization and reinforcement learning domains. We find that lexicase selection with decomposed objectives outperforms MAP-Elites on the deceptive domains that we explore. Furthermore, we find that this technique results in competitive performance on the diversity-focused metrics of QD-Score and Coverage, without explicitly optimizing for these things. We also find that this technique is robust to the choice of subaggregation method and its benefits scale to high dimensional control tasks. This work highlights that diversity does not always need to be explicitly maintained for it to help solve deceptive problems.