Learning dynamical systems provides a new opportunity to tackle a central challenge in science and engineering. 
Model-based approaches show promising results with data samples captured from a single environment. On the other hand, pure data-driven approaches provide plausible results in extracting governing dynamics from multiple environments. The data-driven approach, however, raises a concern about the forecasting of dynamics as recent studies show the limitations of neural networks on extrapolation. 
In this work, we propose sparse regression for multi-environment (SpReME) that can leverage the best of both model-based and data-driven approaches to extract the governing dynamics from multiple environments. We cast the dynamic discovery as a sparse regression problem over multiple environments. The bases of the regression model can be curated with incomplete prior knowledge.
We demonstrate our framework on four different dynamic systems ranging from simple linear to complex chaotic systems. The experimental results show that the extracted knowledge from multiple environments can be generalized to predict the dynamics of an unseen environment.

SpReME: Sparse Regression for Multi-Environment Dynamic Systems

We develop a new measure of the exploration/exploitation trade-off in infinite-horizon reinforcement learning (RL) problems called the occupancy information ratio (OIR), which is comprised of a ratio between the infinite-horizon average cost of a policy and the entropy of its induced long-term state occupancy measure. Modifying the classic RL objective in this way yields policies that strike an optimal balance between exploitation and exploration, providing a new tool for addressing the exploration/exploitation trade-off in RL. The paper develops for the first time policy gradient and actor-critic algorithms for OIR optimization based upon a new entropy gradient theorem, and establishes both asymptotic and non-asymptotic convergence results with global optimality guarantees. In experiments, these methodologies outperform several deep RL baselines in problems with sparse rewards, where many trajectories may be uninformative and skepticism about the environment is crucial to success.

Occupancy Information Ratio: Infinite-Horizon, Information-Directed, Parameterized Policy Search

Learning tasks involving function approximation are prevalent in numerous domains of science and engineering. The underlying idea is to design a learning algorithm that generates a sequence of functions converging to the desired target function with arbitrary accuracy by using the available data samples. In this paper, we present a novel interpretation of iterative function learning through the lens of ensemble dynamical systems, with an emphasis on establishing the equivalence between convergence of function learning algorithms and asymptotic behavior of ensemble systems. In particular, given a set of observation data in a function learning task, we prove that the procedure of generating an approximation sequence can be represented as a steering problem of a dynamic ensemble system defined on a function space. This in turn gives rise to an ensemble systems-theoretic approach to the design of ``continuous-time" function learning algorithms, which have a great potential to reach better generalizability compared with classical ``discrete-time" learning algorithms.

Dynamic Function Learning through Control of Ensemble Systems

Bi-level optimization is a powerful framework to solve a
rich class of problems such as hyper-parameter optimization,
model-agnostic meta-learning, data distillation, and matrix
completion. The existing first-order solutions to bi-level
problems exhibit scalability limitations (for example, in matrix completion) because of the requirement of projecting
solutions onto the feasible set. In this work, we propose a
novel Stochastic Bi-level Frank-Wolfe (SBFW) algorithm
to solve the stochastic bi-level optimization problems in a
projection-free manner. We utilize a momentum-based gradient tracker that results in a sample complexity of O(ϵ−3) for
convex outer objectives with strongly convex inner objectives.
We formulate the matrix completion problem with denoising
as a stochastic bilevel problem and show that SBFW outperforms the state-of-the-art methods for the problem of matrix
completion with denoising and achieves improvements of up
to 82% in terms of the wall-clock time required to achieve the
same level of accuracy

STOCHASTIC BILEVEL PROJECTION-FREE OPTIMIZATION

Many, if not most, systems of interest in science are naturally described as nonlinear dynamical systems (DS). Empirically, we commonly access these systems through time series measurements, where often we have time series from different types of data modalities simultaneously. For instance, we may have event counts in addition to some continuous signal. While by now there are many powerful machine learning (ML) tools for integrating different data modalities into predictive models, this has rarely been approached so far from the perspective of uncovering the underlying, data-generating DS (aka DS reconstruction). Recently, sparse teacher forcing (TF) has been suggested as an efficient control-theoretic method for dealing with exploding loss gradients when training ML models on chaotic DS. Here we incorporate this idea into a novel recurrent neural network (RNN) training framework for DS reconstruction based on multimodal variational autoencoders (MVAE). The forcing signal for the RNN is generated by the MVAE which integrates different types of simultaneously given time series data into a joint latent code optimal for DS reconstruction. We show that this training method achieves significantly better reconstructions on multimodal datasets generated from chaotic DS benchmarks than various alternative methods.

Multimodal Teacher Forcing for Reconstructing Nonlinear Dynamical Systems

To get a good understanding of a dynamical system, it is convenient to have an interpretable and versatile model of it. Timed discrete event systems are a kind of model that respond to these requirements. However, such models can be inferred from timestamped event sequences but not directly from numerical data. To solve this problem, a discretization step must be done to identify events or symbols in the time series. Persist is a discretization method that intends to create persisting symbols by using a score called persistence score. This allows to mitigate the risk of undesirable symbol changes that would lead to a too complex model. After the study of the persistence score, we point out that it tends to favor excessive cases making it miss interesting persisting symbols. To correct this behavior, we replace the metric used in the persistence score, the Kullback-Leibler divergence, with the Wasserstein distance. Experiments show that the improved persistence score enhances Persist's ability to capture the information of the original time series and that it makes it better suited for discrete event systems learning.

Persistence-Based Discretization for Learning Discrete Event Systems from Time Series

We show that stochastic gradient descent (SGD) escapes from sharp minima exponentially fast even before SGD reaches stationary distribution. SGD has been a de facto standard training algorithm for various machine learning tasks. However, there still exists an open question as to why SGDs find highly generalizable parameters from non-convex target functions, such as the loss function of neural networks. An ``escaping'' analysis has been an appealing framework to tackle this question. Escaping analysis measures how quickly SGD  escapes from sharp minima, which is likely to have low generalization abilities. Despite its importance, the framework has the limitation that it works only when SGD reaches a stationary distribution after sufficient updates. In this paper,  we prove that the SGD escapes from sharp minima exponentially fast even in a non-stationary setting. A key tool for the result is the Large Deviation Theory, a fundamental theory in dynamical systems. In particular, we found that a quantity called ``quasi-potential'' is a suitable tool to describe the SGD's stochastic behavior throughout its training process.

Why does SGD prefer flat minima?: Through the lens of dynamical systems

Fluid mechanics continues to advance quickly in the age of artificial intelligence, mainly due to the abundance of experimental data, field data assimilation, and high-fidelity multiscale simulations. Among the many data-driven approaches recently applied to such a discipline, ML-based reduced-order models (ROMs) have received particular attention because of their algorithmic simplicity, explainability, and computational efficiency. In this work, we have devised and implemented an ML-based ROM which combines dimensionality reduction via an Encoder-Decoder (ED) neural network with forecasting capabilities in latent space using Deep Neural Operators (DeepONets). We assessed the proposed architecture with a spatiotemporal dataset generated by the numerical solution of the Rayleigh-Bénard convection (RBC) problem. The reconstruction error of the model over the unseen datasets was lower than $10\%$, demonstrating the ED technique's accurate spatial representation and the neural operators' robustness in estimating future system states. This work represents a solid contribution to the fluid dynamics community with an accurate and efficient ML-based model to tackle the challenging well-known RBC problem.

AAAI 2023 Workshop MLmDS Submissions