Representation and Induction of Finite State Machines using Time-Delay Neural Networks

Daniel S. Clouse, C. Lee Giles, Bill G. Horne, Garrison W. Cottrell

Abstract: This work investigates the representational and inductive capabili(cid:173) ties of time-delay neural networks (TDNNs) in general, and of two subclasses of TDNN, those with delays only on the inputs (IDNN), and those which include delays on hidden units (HDNN) . Both ar(cid:173) chitectures are capable of representing the same class of languages, the definite memory machine (DMM) languages, but the delays on the hidden units in the HDNN helps it outperform the IDNN on problems composed of repeated features over short time windows.