Hands-Free: Action Abstraction With Hierarchical Reinforcement Learning in Text-Based Games

Anjie Zhu, Hongcai He, Yongjun Yang, Zetao Zheng, Jie Shao

Abstract: Large-scale knowledge discovery and decision-making based on textual data are crucial in consumer electronics. For instance, intelligent home robots analyze textual data to tailor household tasks to user preferences, enhancing smart home convenience and comfort. Text-based games offer a research platform for testing and developing artificial intelligence algorithms. These games rely on written narratives and textual interactions while presenting large and combinational action space, and partial observability issues to agents. However, previous methods relied on pre-specifying the subtasks or pre-training on datasets derived from human gameplay, requiring prior knowledge and manual configuration. In this paper, we introduce a novel action abstraction method to free our hands based on the option framework, which uses temporally extended macro-actions to encapsulate relevant long-horizon behaviors. Through leveraging options, our framework decomposes complex tasks autonomously without pre-specified subtasks or demonstrations, thereby partially alleviating the obstacles posed by large and combinatorial action spaces. Moreover, for partial observability issue, we employ a bi-directional attention mechanism to estimate state-action value accurately, strengthening the interdependence between state and action. We conduct extensive experiments on text-based games and the results show the effectiveness of our proposed framework, highlighting its potential in enhancing intelligence in consumer electronics.

External IDs:dblp:journals/tce/ZhuHYZS25