Modified Nelder-Mead Method for High-Dimensional Low-Budget Optimization

Shintaro Takenaga, Yoshihiko Ozaki, Masaki Onishi

Abstract: Black-box optimization (BBO) is a widely used technique for solving a variety of optimization problems including real-world applications with a high-dimensional expensive objective function. Among BBO methods, the Nelder–Mead (NM) method, which is a local search heuristic using a simplex, has been successful due to its simplicity and practical performance on low-dimensional problems. However, the NM method requires <tex>$n+1$</tex> and <tex>$n$</tex> evaluations to perform its initialization and Shrinkage operations respectively to optimize an n-dimensional objective. This is problematic when the objective is computationally and/or financially expensive because, in such a situation, we usually have a limited evaluation budget but those operations consume most of the entire budget. In this study, to address this drawback, we propose a simple but practical modification of the NM method that efficiently works for high-dimensional low-budget optimization. Our numerical results demonstrate that the proposed approach outperforms the original NM method and the random search baselines on BBO benchmark problems.

External IDs:dblp:conf/ssci/TakenagaOO22