Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space

Muhammad Abdullah Naeem, Miroslav Pajic

Keywords: Transportaion inequalities, Exponential Lyapunov function, Sample complexity, Nonlinear random dynamical systems

Abstract:

We study the concentration phenomenon for discrete-time random dynamical systems with an un-
bounded state space. We develop a heuristic approach towards obtaining exponential concentration
inequalities for dynamical systems using an entirely functional analytic framework. We also show
that existence of exponential-type Lyapunov function, compared to the purely deterministic setting,
not only implies stability but also exponential concentration inequalities for sampling from the sta-
tionary distribution, via transport-entropy inequality (T-E). These results have significant impact
in reinforcement learning (RL) and controls, leading to exponential concentration inequalities even
for unbounded observables (i.e., rewards), while neither assuming reversibility nor exact knowledge
of the considered random dynamical system (assumptions at heart of concentration inequalities in
statistical mechanics and Markov diffusion processes).