ClingTP: Curriculum Learning based Multi-style Title Prefix Generation

Yusong Wang, Dongyuan Li, Jialun Shen, Yicheng Xu, Shuai Zhong, Mingkun Xu

Abstract: An informative, creative title prefix is memorable, capable of capturing the attention of readers, and significantly enhances the potential for increased citations. In this work, we pioneer the exploration of the significance of title prefixes in academic papers and propose a controllable title prefix generation model based on curriculum learning. Specifically, we introduce a dedicated dataset named TPOA to compensate for the lack of training data for this emerging task. To make the model capture relevant patterns and language structure, we design three title prefix generation tasks (abstract-based, title-based, and title&abstract-based) to train a ByT5 model into a curriculum learning structure as a generator. After that, we fine-tune another ByT5 model on a target style corpora as a discriminator to control the style of the generated title prefix. Through extensive experiments, our proposed model outperforms existing methods on both human evaluation and automatic evaluation, demonstrating its effectiveness.