Automata Learning from Recurrent Networks: A Critical Synthesis for Verification, Testing, and Interpretability

TMLR Paper7538 Authors

Abstract: Recurrent Neural Networks (RNNs) have demonstrated their effectiveness in modeling sequential data and are a key building block of modern deep learning architectures. In this review paper, we study recurrent networks from the lens of automata theory. Given an RNN, automata learning seeks to model its behavior with an automaton, which enables better interpretability and eases our understanding of its working mechanisms. We begin by examining the theoretical foundations of this approach, displaying how it can be applied to learn automata from various types of recurrent nets, including the Elman Recurrent Network (ERN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). Next, we review the applications of this approach in formal verification, model-based testing, and the interpretability of these deep learning models. We finish with a discussion on the advantages and critical problems of this method, while outlining key goals for future research, such as defining standard benchmarks and identifying limitations that need to be addressed to advance this field further.

Submission Type: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Blake_Aaron_Richards1

Submission Number: 7538