Precise Length Control for Large Language Models
Abstract: Large Language Models (LLMs) have become integral components of production systems, with applications ranging from chatbots like ChatGPT to tasks such as summarisation and question answering. However, a significant challenge with LLMs is the unpredictability of response length, which is particularly problematic in tasks requiring varying levels of detail, such as document summarisation. Here we present a method to adapt existing LLMs to allow control of response length. We achieve this by extending the length-difference positional encoding (LDPE) proposed by (Takase and Okazaki, 2019) to decoder-only transformer architectures. Our approach, termed offset reverse positional encoding (ORPE), uses a positional encoding that counts down from a predetermined response length. Finetuning with ORPE enables the model to learn to structure its responses to terminate at a given length. Our results, obtained from tasks such as question answering and document summarisation, demonstrate that ORPE provides precise control of the response length during inference.
Paper Type: Long
Research Area: Generation
Research Area Keywords: Generation, Summarization, Question Answering, Language Modeling
Contribution Types: NLP engineering experiment
Languages Studied: English
Submission Number: 1378
Loading