Approximate dynamic programming with affine ADDs

Scott Sanner, William T. B. Uther, Karina Valdivia Delgado

Abstract: The Affin ADD (AADD) is an extension of the Algebraic Decision Diagram (ADD) that compactly represents context-specific, additive and multiplicative structure in functions from a discrete domain to a real-valued range. In this paper, we introduce a novel algorithm for efficientl findin AADD approximations that we use to develop the MADCAP algorithm for AADD-based structured approximate dynamic programming (ADP) with factored MDPs. MADCAP requires less time and space to achieve comparable or better approximate solutions than the current state-of-the-art ADD-based ADP algorithm of APRICODD and can provide approximate solutions for problems with context-specific additive and multiplicative structure on which APRICODD runs out of memory.