Global optimization of graph acquisition functions for neural architecture search

ICLR 2026 Conference Submission13065 Authors

Keywords: Graph Bayesian optimization, Mixed-integer programming, neural architecture search, global optimization

TL;DR: We introduce mathematical programming formulations for acquisition functions over graph input spaces for graph BO-based neural architecture search.

Abstract: Graph Bayesian optimization (BO) has shown potential as a powerful and data-efficient tool for neural architecture search (NAS). Most existing graph BO works focus on developing graph surrogate models, i.e., metrics of networks and/or kernels to quantify the similarity between networks. However, optimization of the resulting acquisition functions over graph structures is less studied due to their complexity and formulations over the combinatorial graph search space. This paper presents explicit optimization formulations for graph input spaces, including properties such as reachability and shortest paths, which can then be used to formulate graph kernels and associated acquisition functions. We theoretically prove that the proposed encoding is an equivalent representation of the original graph space and provide a general formulation for neural architecture cells that incorporates node and/or edge-labeled graphs with multiple sources and sinks regardless of connectivity. Numerical results over several NAS benchmarks show that our method efficiently finds the optimal architecture for most cases.

Primary Area: optimization

Submission Number: 13065