Incremental learning with temporary memory

Sanjay Jain, Steffen Lange, Samuel E. Moelius, Sandra Zilles

Abstract: In the inductive inference framework of learning in the limit, a variation of the bounded example memory (Bem<math><mstyle mathvariant="italic" is="true"><mi is="true">Bem</mi></mstyle></math>) language learning model is considered. Intuitively, the new model constrains the learner’s memory not only in how much data may be stored, but also in how long those data may be stored without being refreshed. More specifically, the model requires that, if the learner commits an example x<math><mi is="true">x</mi></math> to memory, and x<math><mi is="true">x</mi></math> is not presented to the learner again thereafter, then eventually the learner forgets x<math><mi is="true">x</mi></math>, i.e., eventually x<math><mi is="true">x</mi></math> no longer appears in the learner’s memory. This model is called temporary example memory (Tem<math><mstyle mathvariant="italic" is="true"><mi is="true">Tem</mi></mstyle></math>) learning.Many interesting results concerning the Tem<math><mstyle mathvariant="italic" is="true"><mi is="true">Tem</mi></mstyle></math>-learning model are presented. For example, there exists a class of languages that can be identified by memorizing k+1<math><mi is="true">k</mi><mo is="true">+</mo><mn is="true">1</mn></math> examples in the Tem<math><mstyle mathvariant="italic" is="true"><mi is="true">Tem</mi></mstyle></math> sense, but that cannot be identified by memorizing k<math><mi is="true">k</mi></math> examples in the Bem<math><mstyle mathvariant="italic" is="true"><mi is="true">Bem</mi></mstyle></math> sense. On the other hand, there exists a class of languages that can be identified by memorizing just one example in the Bem<math><mstyle mathvariant="italic" is="true"><mi is="true">Bem</mi></mstyle></math> sense, but that cannot be identified by memorizing any number of examples in the Tem<math><mstyle mathvariant="italic" is="true"><mi is="true">Tem</mi></mstyle></math> sense.Results are also presented concerning the special case of learning classes of infinite languages.