Controlling Summarization Length Through EOS Token Weighting

Anonymous

Abstract: Controlling the length of generated text can be crucial in various text generation tasks including summarization. Existing methods often require complex model alterations, limiting compatibility with pre-trained models. We address these limitations by developing a simple approach for controlling the length of automatic text summaries by increasing the importance of correctly predicting the EOS token in the cross entropy loss computation. The proposed methodology is agnostic to architecture and decoding algorithm and orthogonal to other inference-time techniques for controlling generation length, allowing for powerful hybrid combinations. We test it with encoder-decoder and modern GPT-style LLMs. We show that our method can consistently control generation length without affecting the quality of the summary.

Paper Type: short

Research Area: Summarization

Contribution Types: NLP engineering experiment

Languages Studied: english