Diffusion-Based Approximate Value FunctionsDownload PDF

Published: 27 Jun 2018, Last Modified: 05 May 2023ICML 2018 ECA SubmissionReaders: Everyone
Keywords: spectral graph theory, geometric deep learning, graph convolutional network, reinforcement learning, deep geometric learning
TL;DR: Solving sparse rewards environments by diffusing the reward signal through Graph Convolutional Networks.
Abstract: We present a novel model-based framework inspired by spectral graph theory and deep geometric learning: the Diffusion-based Approximate Value Function. Our approach efficiently approximates the graph Laplacian of an MDP's underlying graph by using Graph Convolutional Networks (GCN). By generating an approximate value function, we diffuse the reward signal much faster than traditional Reinforcement Learning algorithms such as TD(0). This leads to substantial improvements on sparse rewards environments where efficient credit assignment is most demanding.
3 Replies