\section{Introduction}
Learning from repeated plays in zero-sum games has been a central research problem in game theory since the work of \citep{Brown} and \citep{Robinson}, soon after the appearance of the minimax theorem of von Neumann. In classic normal form zero-sum games, one has to compute probability distributions $\vec{x}_1^*\in\Delta_n$ and $\vec{x}_2^*\in\Delta_m$ that consist an equilibrium of the following problem
\begin{equation*}\label{zs:classic}\tag{Zero-Sum Game}   \max_{\vec{x}_1\in\Delta_m}\min_{\vec{x}_2\in\Delta_n}\vec{x}_1^{\top}A\vec{x}_2
\end{equation*}
where $A$ is an $n\times m$ payoff matrix, and an equilibrium $(\vec{x}_1^*,\vec{x}_2^*)$ is a pair of randomized strategies such that neither player can improve their payoff by unilaterally changing their distributions. 
The dynamics of online learning algorithm in games have been studied extensively. Among a variety of learning methods, Multiplicative Weights Update and Gradient Descent-Ascent, together with their optimistic and extra-gradient variants are of particular interest in \emph{time-independent} games (i.e., the payoff matrix $A$ is time-independent).

Recently, the \textit{last iterate property}, which captures the day-to-day behaviors of learning algorithms in games rather than their average behaviors, has attracted increasing interest due to their wide applications in machine learning and related tasks. In the regime of time-independent games, there have been quite a few results showing the last iterate convergence to Nash equilibrium in zero-sum games. Typical examples include optimistic gradient descent ascent \citep{DISZ17}, extra-gradient descent ascent \citep{LiangS18}  for unconstrained zero-sum games, as well as optimistic multiplicative weights update \citep{daskalakis2018last,fasoulakis2022forward}, extra-gradient multiplicative weights update \citep{mertikopoulos2018optimistic} for constrained zero-sum games.
To conclude, in the context of time-independent games, optimistic methods and extra-gradient methods exhibit similar behaviors : they both possess the last-iterate convergence property and converge by the same rate. Moreover, they can be analyzed in a unified way \citep{mokhtari2020unified}.



Despite aforementioned progresses on time-independent games, only recently there have emerged researches on learning in time-varying zero-sum games \citep{cardoso2019competing, fiez2021online,Duvocelle18:Multi,zhou2022,anagnostides2023convergence, feng2023last}. 
In particular, it has been established by \citep{feng2023last} that the optimistic gradient descent-ascent and extra gradient descent ascent have fundamentally different last iterate behaviors, unlike previous studies in time-independent zero-sum games. Nevertheless, \citep{feng2023last} focuses on the setting of \textit{unconstrained} zero-sum games. However, compared to unconstrained games, games with constrains are more common both in practical and theoretical studies. In this paper, we aim to address the following question:

\textit{Is there a similar last-iterate convergence separation between optimistic and extra-gradient methods in constrained time-varying games ?}

\paragraph{Our contribution.} We highlight the following two results as our main contribution :
\begin{itemize}
    \item We construct a constrained periodic game with a common equilibrium and prove optimistic multiplicative weights update do not converge to the equilibrium in this game. See 
    Theorem \ref{thm: OMWU fails}.
    \item We prove that if the game series in a periodic game with simplex constrains have a common equilibrium, then Extra-gradient multiplicative weights update will converge to this equilibrium. See Theorem \ref{T2}. 
\end{itemize}

By combining these two terms, we prove that there exist a clear last-iterate convergence separation between optimistic and extra-gradient methods in constrained periodic games, thereby extending the results of \citep{feng2023last} from unconstrained to constrained settings.

\paragraph{Technical Comparison.}

    The MWU-based algorithms considered in this paper differ from the GDA-based algorithms considered in \citep{feng2023last} in two fundamental ways. Firstly, variations of MWU algorithms are naturally defined on the simplex constraints, allowing our analysis to avoid the difficulty of projecting onto simplex. Secondly, the algorithms considered in \citep{feng2023last} have linear structure, i.e., can be directly analyzed as linear systems, while the MWU-based algorithms have non-linear, making the techniques of \citep{feng2023last} ineffective in our scenario. At a high level, by considering variations of MWU algorithms, we transform the technical difficulties arising from constraints into difficulties related to analyzing non-linear dynamics of MWU-based algorithms. It is worth noting that a similar transformation can be observed in the line of research on establishing last-iterate convergence in static games: \citep{daskalakis2017training} first proved convergence of Optimistic GDA without constraints and then \citep{daskalakis2018last} extended their results to constrained settings for Optimistic MWU. 

%    To be more specific, allow us to compare the proof of Theorem 3.2 in our paper with Theorem 3.1 in \citep{feng2023last}. Note that both theorems demonstrate that Extra-gradient methods converge in periodic games.  
    
%    In \citep{feng2023last}, the authors rely on a key fact that characterizes the iterative matrices of Extra-GDA as normal matrices with eigenvalues not exceeding 1 (Lemma A.2 and B.1 in their paper). This simplifies the problem by leveraging existing tools in linear dynamical systems. A natural extension of their approach to Extra-MWU algorithms would be to prove similar results for the Jacobian matrix of Extra-MWU near the equilibrium and as a result provide guarantees in a small neighborhood around the equilibrium. Nevertheless, %even if this generalization holds, 
%    this does not imply the global convergence property of Extra-MWU in periodic games, due to its non-linearity; the Jacobian matrix can only describe behaviors of points close to equilibrium, resulting in a local convergence result for Extra-MWU (local guarantees). To obtain a global convergence result, we employ a Lyapunov-type argument specifically tailored for Extra-MWU (Section 4.2 in our paper), and this differs significantly from \citep{feng2023last}. A similar difficulty also appears in establishing the divergence result of Optimistic MWU.
    

\paragraph{Organization.} In Section \ref{pre}, we present the necessary background for this work. In Section \ref{main}, we state our main results. In Section \ref{opop}, we explain the main ideas behind the proof of our theoretical results. In Section \ref{exexex},  we provide numerical experiments to support our theoretical findings. Discussions on possible extensions of the current results are presented in Section \ref{dc}.