Focus on this, not that! Steering LLMs with Adaptive Feature Specification

Tom A. Lamb, Adam Davies, Alasdair Paren, Philip Torr, Francesco Pinto

Keywords: Instruction Tuning, LLMs, Spurious Correlations

TL;DR: We introduce Focus Instruction Tuning (FIT), a method to trains LLMs to adaptively condition their task behaviours based on specified features, inducing steerability and controllability through feature specification.

Abstract: Despite the success of Instruction Tuning (IT) in training large language models (LLMs) to perform arbitrary user-specified tasks, these models often still leverage spurious or biased features learned from their training data, leading to undesired behaviours when deploying them in new contexts. In this work, we introduce Focus Instruction Tuning (FIT), which trains LLMs to condition their responses by focusing on specific features whilst ignoring others, leading to different behaviours based on what features are specified. Across several experimental settings, we show that focus-tuned models can be adaptively steered by focusing on different features at inference-time: for instance, robustness can be improved by focusing on core task features and ignoring spurious features, and social bias can be mitigated by ignoring demographic categories. Furthermore, FIT can steer behaviour in new contexts, generalising under distribution shift and to new unseen features at inference time, and thereby facilitating more robust, fair, and controllable LLM applications in real-world environments.

Submission Number: 90