Adaptive Decision-Making in Non-Stationary Markov Decision Processes

Baiting Luo

Abstract: This research addresses a critical and largely unresolved challenge in the field of sequential decision-making: operating effectively in non-stationary environments. These environments are characterized by exogenously-driven changes over time, introducing significant uncertainties in decision-making processes. The urgency lies in devising strategies for optimal decision-making and planning amidst these unpredictable conditions. Central to my research is the concept of 'anytime' decision-making. This approach involves leveraging dynamically learned models that not only mirror the current environmental state but also anticipate its potential evolution. The focus is on how an agent adapts its decision-making process in an ever-changing environment. A key contribution of my work is the exploration of adaptive decision-making strategies employed by an agent whose objectives fluctuate between performance optimization and safety prioritization. This is particularly challenging in dynamic environments where traditional static decision-making models fall short. The paper concludes by presenting future research directions. These aims are to enhance the understanding of adaptive decision-making in non-stationary environments, thereby advancing the field in this complex and constantly evolving area.