Remember what you did so you know what to do next

Manuel Rafael Ciosici, Alex Hedges, Yash Kankanampati, Justin Andrew Martin, Marjorie Freedman, Ralph M. Weischedel

Submission Type: Regular Short Paper

Submission Track: Commonsense Reasoning

Submission Track 2: Language Modeling and Analysis of Language Models

Keywords: large language models, text games

TL;DR: We present a new ScienceWorld result that is 3.5x higher than that of reinforcement learning and matches some of the results of GPT-3-based architectures.

Abstract: We explore using the 6B parameter GPT-J language model to create a plan for a simulated robot to achieve 30 classes of goals in ScienceWorld, a text game simulator for elementary science experiments and for which previously published empirical work has shown large language models (LLM)s to be a poor fit (Wang et al., 2022). Using the Markov assumption, the LLM outperforms the state-of-the-art based on reinforcement learning by a factor of 1.4. When we fill the LLM’s input buffer with as many prior steps as will fit, improvement rises to 3.3x. Even when training on only 6.5% of the training data, we observe a 2.3x improvement over the state-of-the-art. Our experiments show that performance varies widely across the 30 classes of actions, indicating that averaging over tasks can hide significant performance issues.

Submission Number: 4126