Robust Quadratic Programming for MDPs with uncertain observation noise

Jianmei Su, Hong Cheng, Hongliang Guo, Zhinan Peng

Abstract: The problem of Markov decision processes (MDPs) with uncertain observation noise has rarely been studied. This paper proposes a Robust Quadratic Programming (RQP) approach to approximate Bellman equation solution. Besides efficiency, the proposed algorithm exhibits great robustness against uncertain observation noise, which is essential in real world applications. We further represent the solution into kernel forms, which implicitly expands the state-encoded feature space to higher or even infinite dimensions. Experimental results well justify its efficiency and robustness. The comparison with different kernels demonstrates its flexibility of kernel selection for different application scenarios.