DOM-Q-NET: Grounded RL on Structured Language

Sheng Jia, Jamie Ryan Kiros, Jimmy Ba

Sep 27, 2018 ICLR 2019 Conference Blind Submission readers: everyone Show Bibtex
  • Abstract: The ability for agents to interact with the web would allow for significant improvements in knowledge understanding and representation learning. However, web navigation tasks are difficult for current deep reinforcement learning (RL) models due to large discrete action space and varying number of actions between the states. In this work, we introduce DOM-Q-NET, a novel architecture for RL-based web navigation to address both of these problems. It parametrizes Q functions with separate networks for different action categories, clicking DOM and typing a string input. DOM-Q-NET utilizes a graph neural network to represent the tree-structured HTML of a standard web page. We demonstrate the capabilities of our model on the WorldOfBits(MiniWoB) environment where we can match or outperform existing work without the use of expert demonstrations. Furthermore, we show 2x improvements in sample efficiency when training in the multitask setting, allowing our model to transfer learned behaviours across tasks.
  • Keywords: Reinforcement Learning, Web Navigation, Graph Neural Networks
0 Replies