Multilevel Approach to Efficient Gradient Calculation in Stochastic Systems

Joohwan Ko, Michael Poli, Stefano Massaroli, Woo Chang Kim

TL;DR: We propose an efficient gradient calculation approach in stochastic systems using a multilevel method, which reduces computational cost and enhances the training of the model.

Abstract: Gradient estimation in Stochastic Differential Equations is a critical challenge in fields that require dynamic modeling of stochastic systems. While there have been numerous studies on pathwise gradients, the calculation of expectations over different realizations of the Brownian process in SDEs is occasionally not considered. Multilevel Monte Carlo offers a highly efficient solution to this problem, greatly reducing the computational cost in stochastic modeling and simulation compared to naive Monte Carlo. In this study, we utilized Neural Stochastic Differential Equations as our stochastic system and demonstrated that the accurate gradient could be effectively computed through the use of MLMC.