Towards Understanding Semantic Degeneration in Text-Based Reinforcement Learning

Anonymous

Abstract: The typical text-based reinforcement learning setting involves an agent interacting with a fictional environment using textual information and commands to complete some task. Past work has shown that agents are able to succeed in text-based tasks even in the complete absence of semantic understanding or other linguistic capabilities. Most agents rely entirely or almost entirely on language representations learned from a single game. The success of these agents in playing such games suggest that semantic understanding not important to the task. In this work, we describe the occurrence of semantic degeneration as a consequence of inappropriate fine-tuning language models in text-based reinforcement learning. Our results show that fine-tuning large language models may lead to poor performance. In addition, we show that, even though semantics is not required for successful training, a semantically rich representation improves the generalization of these agents.

Paper Type: short

Research Area: Dialogue and Interactive Systems

Languages Studied: English