QFMTS: Generating Query-Focused Summaries over Multi-Table Inputs

Anonymous

Abstract: Query-focused summarization (QFS) has been well-studied in the context of text-based data, However, QFS over semi-structured data such as tables remains under-explored. Existing studies primarily focus on single-table context, thus limiting the capability to handle complex multi-table scenarios. In this paper, we introduce a novel query-focused multi-table summarization task (QFMTS), where generation models should produce comprehensive query-focused summaries from multi-table contexts. This requires the models to perform arithmetic and multi-table operations such as join and intersect. To facilitate this task, we automatically collect the QFMTS dataset by leveraging large language models (LLMs) as data annotators. The dataset consists of 6,404 query-summary pairs, each accompanied by multiple tables. Our quality evaluation, including automatic and human evaluation, illustrates the high quality of the dataset. To demonstrate the efficacy of the dataset, we experiment with state-of-the-art models, including open-source generation models and closed LLMs, on QFMTS. Experiment results and qualitative analysis reveal the significant challenges of the proposed task.

Paper Type: long

Research Area: Summarization

Contribution Types: Data resources

Languages Studied: English

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.