Inversion in Time

Sebastian Thrun, Alexander Linden

Abstract: Inversion of multilayer synchronous networks is a method which tries to answer questions like “What kind of input will give a desired output?” or “Is it possible to get a desired (output under special input/output constraints)?”. We will describe two methods of inverting a connectionist network. Firstly, we extend inversion via backpropagation (Linden/Kindermann [4], Williams [11]) to recurrent (Elman [1], Jordan [3], Mozer [5], Williams/Zipser [10]), time-delayed (Waibel at al. Secondly, we introduce a new inversion method for proving the non-existence of an input combination under special constraints, e.g. in a subspace of the input space. This method works by iterative exclusion of invalid activation values. It might be a helpful way to judge the properties of a trained network. We conclude with simulation results of three different tasks: XOR, morse signal decoding and handwritten digit recognition.