% !TEX root = main.tex

\begin{abstract}
Recently, multi-objective optimization (MOO) has gained attention for its broad applications in ML, operations research, and engineering.
%
However, MOO algorithm design remains in its infancy and many existing MOO methods suffer from unsatisfactory convergence rate and sample complexity performance.
%
To address this challenge, in this paper, we propose an algorithm called \alg (\ul{st}ochastic path-\ul{i}ntegrated \ul{mul}ti-gradient rec\ul{u}rsive e\ul{s}timator), a new and robust approach for solving MOO problems. 
%
Different from the traditional methods,
\alg introduces a simple yet powerful recursive framework for updating stochastic gradient estimates to improve convergence performance with low sample complexity.
%
In addition, we introduce an enhanced version of \algns, termed \algmns, which incorporates a momentum term to further expedite convergence.
%
We establish $\mathcal{O}(1/T)$ convergence rates of the proposed methods for non-convex settings and $\mathcal{O}(\exp{-\mu T})$ for strongly convex settings, where  $T$ is the total number of iteration rounds. Additionally, we achieve the state-of-the-art $O\left(n+\sqrt{n}\epsilon^{-1}\right)$  sample complexities for non-convex settings and $\mathcal{O}\left(n+ \sqrt{n} \ln ({\mu/\epsilon})\right)$ for strongly convex settings, where $\epsilon>0$ is a desired stationarity error.
Moreover, to alleviate the periodic full gradient evaluation requirement in \alg and \algmns, we further propose enhanced versions with adaptive batching called \algpns/ \algmp and provide their theoretical analysis.
%
\iffalse

Recently, multi-objective optimization (MOO) problems have received increasing attention due to their wide range of applications in various fields, such as machine learning (ML), operations research, and many engineering applications. However, MOO algorithm design remains in its infancy and many existing MOO methods suffer from unsatisfactory convergence rate and sample complexity performance. To address this challenge, in this paper, we propose an algorithm called STIMULUS (**st**ochastic path-**i**ntegrated **mul**ti-gradient rec**u**rsive e**s**timator), a new and robust approach for solving MOO problems. Different from the traditional methods,
STIMULUS introduces a simple yet powerful recursive framework for updating stochastic gradient estimates to improve convergence performance with low sample complexity. In addition, we introduce an enhanced version of \algns, termed \algmns, which incorporates a momentum term to further expedite convergence. 

We establish $\mathcal{O}(1/T)$ convergence rates of the proposed methods for non-convex settings and $\mathcal{O}(\exp{-\mu T})$  for strongly convex settings, where  $T$ is the total number of iteration rounds. Additionally, we achieve the state-of-the-art $O\left(n+\sqrt{n}\epsilon^{-1}\right)$  sample complexities for non-convex settings and $\mathcal{O}\left(n+ \sqrt{n} \ln ({\mu/\epsilon})\right)$ for strongly convex settings, where $\epsilon>0$ is a desired stationarity error.

Moreover, to alleviate the periodic full gradient evaluation requirement in STIMULUS and STIMULUS-M, we further propose enhanced versions with adaptive batching called STIMULUS$^+$/ STIMULUS-M$^+$ and provide their theoretical analysis.
\fi
\end{abstract}