An Evolutionary Transfer Reinforcement Learning Framework for Multiagent Systems

Yaqing Hou, Yew-Soon Ong, Liang Feng, Jacek M. Zurada

Abstract: In this paper, we present an evolutionary transfer reinforcement learning framework (eTL) for developing intelligent agents capable of adapting to the dynamic environment of multiagent systems (MASs). Specifically, we take inspiration from Darwin's theory of natural selection and Universal Darwinism as the principal driving forces that govern the evolutionary knowledge transfer process. The essential backbone of our proposed eTL comprises several meme-inspired evolutionary mechanisms, namely meme representation, meme expression, meme assimilation, meme internal evolution, and meme external evolution. Our proposed approach constructs social selection mechanisms that are modeled after the principles of human learning to identify appropriate interacting partners. eTL also models the intrinsic parallelism of natural evolution and errors that are introduced due to the physiological limits of the agents' ability to perceive differences, so as to generate “growth” and “variation” of knowledge that agents have of the world, thus exhibiting higher adaptivity capabilities on solving complex problems. To verify the efficacy of the proposed paradigm, comprehensive investigations of the proposed eTL against existing state-of-the-art TL methods in MAS, are conducted on the “minefield navigation tasks” platform and the “Unreal Tournament 2004” first person shooter computer game, in which homogeneous and heterogeneous learning machines are considered.