Reinforcement Learning in Continuous Spaces by Using Learning Fuzzy Classifier Systems

Gang Chen, Colin Douch, Mengjie Zhang, Shaoning Pang

Abstract: Aimed at achieving multi-step reinforcement learning in continuous spaces, many Learning Classifier Systems have been developed recently to learn fuzzy logic rules. Among these systems, accuracy-based Michigan learning fuzzy classifier systems are gaining increasing research attention. However, in order to learn effectively, existing accuracy-based systems often require the action space to be discrete. Without this restriction, only single-step learning may be supported. In this paper, we will develop a new accuracy-based learning fuzzy classifier system that can perform multi-step reinforcement learning in completely continuous domains. To achieve this goal, a special fuzzy logic system will be introduced in this paper where the output action from the system is modelled through a continuous probability distribution. A natural gradient learning technique will be further exploited to fine-tune the action outputs of individual fuzzy rules. The effectiveness of our learning system has been verified on several benchmark problems.