The Blessing of Dimensionality in Many-Objective Search: An Inverse Machine Learning Insight

Abhishek Gupta, Yew-Soon Ong, Mojtaba Shakeri, Xu Chi, Allan NengSheng Zhang

Abstract: Sample-based evolutionary algorithms (EAs) are widely used for optimizing problems with multi (greater than one but less than four) or even many (greater than or equal to four) objectives of interest. In general, the difficulty of a problem exponentially increases with the number of objectives, serving as a clear example of the curse of dimensionality. The exploratory approach an EA takes in these cases has led to it being thought of as a big data generator, progressively sampling and evaluating solutions in high performing regions of a decision space to guide the search towards optimal solutions. Notably, in both multi- and many-objective EAs, the sampled data can be further utilized for building inverse generative models, mapping points in objective space back to solutions in the decision space. Such models offer immense flexibility to a decision maker in generating new target solutions on the fly, thereby facilitating real-time a posteriori preference incorporation into the search. In this paper, we show that the data distribution resulting from a many-objective formulation is in fact more conducive to building accurate inverse models than its multiobjective counterpart. Given the potential utility of these models, we in turn shed light on a rare blessing of dimensionality that is yet to be explored in the context of optimization. We first present simple theoretical arguments supporting our claim. Thereafter, experimental studies of Gaussian process-based inverse modeling for a synthetic and a real-world example are carried out to further confirm the theory.