Guided Zeroth-Order Methods for Stochastic Non-convex Problems with Decision-Dependent Distributions

Yuya Hikima, Hiroshi Sawada, Akinori Fujino

TL;DR: Improved zeroth-order methods for stochastic non-convex optimization problems with decision-dependent distributions

Abstract:

In this study, we tackle an optimization problem with a known function and an unknown decision-dependent distribution, which arises in a variety of applications and is often referred to as a performative prediction problem.
To solve the problem, several zeroth-order methods have been developed because the gradient of the objective function cannot be obtained explicitly due to the unknown distribution.
Although these methods have theoretical convergence, they cannot utilize the information on the known function, which limits their efficiency in reducing the objective value.
To overcome this issue, we propose new zeroth-order methods that generate effective update directions by utilizing information on the known function.
As theoretical results, we show the convergence of our methods to stationary points and provide the worst-case sample complexity analysis, which improves the state of the arts when the maximum objective value dominates the cube root of the decision variable's dimensionality in order.
Our simulation experiments on multiple applications show that our methods output solutions with lower objective values than the existing zeroth-order methods do.

Lay Summary: In this study, we address optimization problems in which the underlying probability distribution depends on the decision variable. Such problems arise in various real-world applications. For instance, in finance, a lender may want to train a classifier with parameter $x$ to identify reliable customers. However, the distribution of customer features can change depending on the parameter $x$, since customers might adjust their features in response to the classifier’s behavior. This interaction creates the need to solve an optimization problem where the distribution is influenced by the decision variable—known as a decision-dependent distribution. To tackle this challenge, we propose new zeroth-order optimization methods that effectively construct update directions by leveraging information from the known objective function.

Link To Code: https://github.com/Yuya-Hikima/ICML2025-Guided-Zeroth-Order-Methods/tree/main

Primary Area: Optimization->Zero-order and Black-box Optimization

Keywords: Zeroth-order method, performative prediction, decision-dependent distribution, guided evolutionary strategy

Flagged For Ethics Review: true

Submission Number: 1660