Particle Filter Network: A Model-free Approach for POMDP

Pengfei Gao, Junzi Zhang

Abstract: In this paper, we consider the problem of model-free POMDP learning without knowing the hidden state transition dynamics, either in the form of probability models or simulators. We propose to solve it using a neural-network based particle filter, and combine it with an LSVI planning module with a Q-value network. Our preliminary experiments show that the particle filter network can be well-learned without the decision making part. It remains to be studied how the inclusion of the planning module will perform when nontrivial decision making is involved.