Distributionally Constrained Stochastic Gradient Estimation Using Noisy Function Evaluations

Henry Lam, Junhui Zhang

Abstract: We consider gradient estimation with only noisy function evaluation, where the function can only be evaluated at values lying within a probability simplex. We are interested in obtaining gradient estimators where each (pair of) data collection or simulation run applies simultaneously to all directions at once. Our problem is motivated from the use of stochastic approximation in distributionally robust simulation analysis, which involves solving for worst-case input distributions in a black-box simulation model. In this context, conventional gradient schemes such as simultaneous perturbation face challenges as the required moment conditions that allow the "canceling" of higher-order error terms cannot be satisfied without violating the simplex constraints. We investigate a new set of required conditions on the probability distribution that governs the perturbation, which leads us to a class of implementable gradient estimators using Dirichlet mixtures. We study the statistical properties of these estimators and demonstrate their effectiveness with numerical results.