Time-series Forecasting Coding: A New Processing Method Developed from Predictive Coding for Recurrent Neural Networks

Yuto Wakui, Junya Kato, Ryo Natsuaki, Akira Hirose

Abstract: Recently, recurrent neural networks (RNNs) and the free energy principle (FEP) have been attracting much attention. FEP includes the so-called predictive coding (PC), in which the RNN input signals are coded as the difference between the RNN prediction output signals and original input signals. This paper proposes a new method for processing time-series data in RNNs, namely, time-series forecasting coding (TFC), developed by being based on the idea of PC. We consider multi-step ahead forecasting task with the following two schemes in our proposed TFC, i.e., smallest-delay feedback TFC and synchronous (temporally matching) feedback TFC. The latter is a natural extension of conventional PC. Experimental results show that the smallest-delay TFC presents the highest performance, while synchronous TFC shows comparable or lower performance, and then the conventional method indicates the lowest. This result suggests that the concept of PC is important even in time-series data processing, and that small delay is more important than synchronicity. This may have significant implications to the conceptual fundamentals of the PC and thus of the FEP.