A qualitative theory of dynamical systems for assessing stability in ResNets

Woochul Jung, Pildong Hwang, C.A. Morales, Jong-Yeup Kim, SEUNGHEE LEE

Primary Area: visualization or interpretation of learned representations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Dynamical Systems, stability, residual neural networks

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Assessing stability in ResNets via the theory of Dynamical Systems

Abstract:

We present an experimental method for evaluating the stability of ResNets, inspired by the qualitative theory of dynamical systems. To apply qualitative and quantitative properties from the literature on dynamical systems, we have proposed ResNets designed to maintain dimensionality throughout the residual blocks. As a result, we can not only introduce a well-suited concept of expansivity and shadowing properties for ResNets but also analyze their numerical degrees based on Dynamical Systems theory. This work aims to contribute to the understanding of ResNets' stability and bridge the gap between theory and practical applications.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5010