Generalizing Objective-Specification in Markov Decision Processes

Pedro P. Santos

Abstract: In this thesis, we address general utility Markov decision processes (GUMDPs), which generalize the standard Markov decision processes (MDPs) framework for decision-making by considering a broader range of objective functions that depend on the occupancy induced by a given policy. We aim to study GUMDPs from a theoretical perspective and develop new algorithms to solve GUMDPs by leveraging optimization techniques. We also aim to better understand how objective specification in GUMDPs compares to that of MDPs, further studying the connections between the two frameworks for sequential decision-making. We hope that, by achieving the proposed goals, the contributions of this thesis can lay down the foundations supporting the future development and deployment of agents that take advantage of the diverse set of objectives that can be encoded with GUMDPs.