Enhancing User Understanding of Reinforcement Learning Agents Through Visual Explanations

Yotam Amitai

Abstract: With the rapid advancement of Artificial Intelligence, the frequency of interaction between people and autonomous agents is on the rise. Effective human-agent collaboration requires that people understand the agent's behavior. Failing to do so may cause reduced productiveness, misuse, frustration, and even danger. Current explainable AI methods prioritize interpreting the local decisions of an agent, putting less emphasis on the challenge of conveying global behavior. Furthermore, there is a growing demand for explanation methods for agents in sequential decision-making frameworks such as reinforcement learning. Agent strategy summarization methods are used to describe the strategy of an agent to its user through demonstration. The summary's purpose is to maximize the user's understanding of the agent's aptitude by showcasing its behavior in a set of world states, chosen by some importance criteria. Extracting the crucial states from the execution traces of the agent in such a way as to best portray the agent's behavior is a challenging task. My thesis tackles this objective by adding to the equation the context in which the user interacts with the agent. This research proposes novel methods for generating summary-based explanations for reinforcement learning agents