The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment'' in Large Language Models

Hannah Kirk, Bertie Vidgen, Paul Rottger, Scott Hale

Keywords: alignment, large language models, human feedback learning, RLHF, socio-political theory

TL;DR: Using post-structuralist socio-political theory, we address the challenge of translating the abstract concept of "alignment" into empirical signals and offer a framework for consistent communication across various empirical paradigms.

Abstract: In this paper, we address the concept of ``alignment'' in large language models (LLMs) through the lens of post-structuralist socio-political theory, specifically examining its parallels to empty signifiers. To establish a shared vocabulary around how abstract concepts of alignment are operationalised in empirical datasets, we propose a framework that demarcates: 1) which dimensions of model behaviour are considered important, then 2) how meanings and definitions are ascribed to these dimensions, and by whom. We situate existing empirical literature and provide guidance on deciding which paradigm to follow. Through this framework, we aim to foster a culture of transparency and critical evaluation, aiding the community in navigating the complexities of aligning LLMs with human populations.

Submission Number: 64