Online Reference Tracking For Linear Systems with Unknown Dynamics and Unknown Disturbances
Abstract: This paper presents an online learning mechanism to address the challenge of state tracking for unknown linear systems under general adversarial disturbances. The reference trajectory is assumed to be generated by unknown exosystem dynamics, which relaxes the common assumption of known dynamics for exosystems. Learning a tracking control policy for unknown systems with unknown exosystem dynamics under general disturbances is challenging and surprisingly unsettled. To face this challenge, the presented online learning algorithm has two stages: In the first stage, an algorithm identifies the dynamics of the uncertain system, and in the second stage, an online parametrized memory-augmented controller accounts for the identification error, unknown exosystem dynamics as well as disturbances. The controller's parameters are learned to optimize a general convex cost function, and learning the control parameters is formulated as an online convex optimization problem. This approach uses the memory of previous disturbances and reference values to capture their effects on performance over time. Besides, it implicitly learns the dynamics of the exosystems. The algorithm enables online tuning of controller parameters to achieve state tracking and disturbance rejection while minimizing general convex costs. It is shown that the algorithm achieves a policy regret of $\mathcal{O}({T}^{\frac{2}{3}})$. In the simulation results, the performance of the presented tracking algorithm was compared with the certainty equivalent $H_{\infty}$-control and linear quadratic regulator.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: We have tried to address the comments of the reviewers, including but not limited to adding a real-world application of the proposed tracking algorithm, fixing the typos, and modifying the order of the material to highlight the technical novelty of our work.
Assigned Action Editor: ~Aleksandra_Faust1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 1414
Loading