A Fast and Effective Solution to the Problem of Look-ahead Bias in LLMs

Humzah Merchant, Bradford Levy

Keywords: Unlearning, Large Language Models, Look-ahead bias, forecasting

TL;DR: We show that using two small models to guide generation of a large model at inference time is an effective and nearly free method of unlearning.

Abstract: Applying LLMs to predictive tasks in finance is challenging due to look-ahead bias resulting from their training on long time-series data. This precludes the backtests typically employed in finance since retraining frontier models from scratch with a specific knowledge cutoff is prohibitive. In this paper, we introduce a fast, effective, and low-cost alternative. Our method guides generation at inference time by adjusting the logits of a large base model using a pair of smaller, specialized models—one fine-tuned on information to be forgotten and another on information to be retained. We demonstrate that our method effectively removes both verbatim and semantic knowledge, corrects biases, and outperforms prior methods.

Submission Number: 113