Learning to Reduce: Towards Improving Performance of Large Language Models on Structured Data

Younghun Lee, Sungchul Kim, Ryan A. Rossi, Tong Yu, Xiang Chen

Keywords: Structured data QA, Table QA, Large Language Models, LLM Prompting, Reinforcement Learning, Policy Optimization

TL;DR: RL-based tuning of T5 can better generate reduced version of structured data (e.g. tables), improving GPT-4's inference on Table QA tasks.

Abstract:

Large Language Models (LLMs) have been achieving competent performance on a wide range of downstream tasks, yet existing work shows that inference on structured data is challenging for LLMs. This is because LLMs need to either understand long structured data or select the most relevant evidence before inference, and both approaches are not trivial.
This paper proposes a framework, Learning to Reduce, that fine-tunes a language model with On-Policy Learning to generate a reduced version of an input structured data. When compared to state-of-the-art LLMs like GPT-4, Learning to Reduce not only achieves outstanding performance in reducing the input, but shows generalizability on different datasets. We further show that the model fine-tuned with our framework helps LLMs better perform on table QA tasks especially when the context is longer.

Submission Number: 30