Non-Parametric Optimality Conditions for Stochastic Opti- mizations with Stochastic Decision

Mohsen Amidzadeh, Mario Di Francesco

Abstract:

Stochastic optimization (SO) plays a central role in addressing decision‐making problems under uncertainty.
Among them, time-varying stochastic optimization (TV-SO)
%and functional stochastic optimization (F-SO)
is particularly an paramount class of SO problems.
Non-parametric optimality has been studied for the time-varying
%and functional
\emph{deterministic} optimizations, however, it has not been evaluated for their stochastic counterparts. This work specifically addresses non-parametric optimality by developing a stochastic variational framework based on Malliavin calculus.
This allows to derive non-parametric optimality conditions for considered SO problem
with a stochastic decision and to devise a scalable deep-learning algorithm insensitive to the parameterization dimension.
Such an algorithm, called stochastic path follower (SPF), is applied to solve two important problems under distribution drift,
namely, least-squares recovery and logistic regression.
The experimental results show the merit of the proposed solution
against learning-based and gradient-based state-of-the-art methods from performance and scalability perspectives.