Game of Thought: Robust Information Seeking with Large Language Models Using Game Theory

Langyuan Cui, Hwee Tou Ng, Chun Kai Ling

Keywords: Game Theory, Information Seeking, LLMs

TL;DR: We formalize the strategic language search problem and propose game theoretical methods for solving it.

Availability: If accepted, I will present in person

Abstract: Large Language Models (LLMs) are increasingly deployed in real-world scenarios where they may lack sufficient information to complete a given task. In such settings, the ability to actively seek out missing information becomes a critical capability. However, existing approaches to enhancing this ability often rely on simplifying assumptions that degrade \textit{worst-case} performance. This is an issue with serious implications in high-stakes applications. In this work, we introduce and formalize the Strategic Language Search (SLS) problem along with its variants as a two-player zero-sum extensive form game. This formulation provides a principled framework for evaluating the information-seeking capability of LLMs. We propose Game of Thought (GoT), a simple yet effective framework that applies game-theoretic techniques to approximate a Nash equilibrium strategy for the restricted variant of the game. Empirical results demonstrate that our approach consistently improves worst-case performance compared to (1) direct prompting-based methods and (2) heuristic-guided search methods across all tested settings.

Submission Number: 15