ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Context

Anonymous

Abstract: Tables play a crucial role in conveying information in various domains, serving as indispensable tools for organizing and presenting data in a structured manner. We propose a \textit{Plan-then-Reason} framework to answer different types of user queries over tables with sentence context. The framework first plans the reasoning paths over the context, then assigns each step to program-based or textual reasoning to reach the final answer. We construct an instruction tuning set \texttt{TrixInstruct} following the framework. Our dataset cover queries that are program-unsolvable or need combining information from tables and sentences to obtain planning and reasoning abilities. We present \textsc{ProTrix} by finetuning Llama-2-7B on \texttt{TrixInstruct}. Our experiments show that \textsc{ProTrix} generalizes to diverse tabular tasks and achieves comparable performance to GPT-3.5-turbo. We further demonstrate that \textsc{ProTrix} can generate accurate and faithful explanations to answer complex free-form questions. Our work underscores the importance of the planning and reasoning abilities towards a model over tabular tasks with generalizability and interpretability. We will release our dataset and model to the research community.

Paper Type: long

Research Area: NLP Applications

Contribution Types: NLP engineering experiment, Publicly available software and/or pre-trained models

Languages Studied: English