Reinforcement Learning Algorithms: Categorization and Structural Properties

Kenneth Schröder, Alexander Kastius, Rainer Schlosser

Abstract: Over the last years, the field of artificial intelligence (AI) has continuously evolved to great success. As a subset of AI, Reinforcement Learning (RL) has gained significant popularity as well and a variety of RL algorithms and extensions have been developed for various use cases. Although RL is applicable to a wide range of problems today, the amount of options is overwhelming and identifying the advantages and disadvantages of methods for selecting the most suitable algorithms is difficult. Sources use conflicting terminology, imply improvements to alternative algorithms without mathematical or empirical proof, or provide incomplete information. As a result, there is the chance for engineers and researchers to miss alternatives or perfect-fit algorithms for their specific problems. In this paper, we identify and explain essential properties of RL problems and algorithms. Our discussion of these concepts can be used to select, optimize, and compare RL algorithms and their extensions with respect to particular problems, as well as reason about their performance.