Learning to Plan and Generate Text with Citations

Constanza Fierro, Reinald Kim Amplayo, Fantine Huot, Nicola De Cao, Joshua Maynez, Shashi Narayan, Mirella Lapata

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: summarization, text generation, content planning, attribution

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

The increasing demand for the deployment of LLMs in
information-seeking scenarios has spurred efforts in creating
verifiable systems, which generate responses to queries along with
supporting evidence. In this paper, we explore the attribution
capabilities of plan-based models which have been recently shown to
improve the faithfulness, grounding, and controllability of generated
text. We conceptualize plans as a sequence of questions which serve as
blueprints of the generated content and its organisation. We
experiment with two models that utilize different variants of
blueprints, an abstractive model where questions are
generated from scratch, and an extractive
model where the decoder is forced to copy questions from the
input. Experiments on long-form question-answering show
that output quality improves for blueprint models when these learn
to generate responses with attribution. Moreover, the citations generated by blueprint models are more accurate compared to those obtained from
LLM-based pipelines lacking a planning component.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2736