Leveraging LLM-Generated Schema Descriptions for Unanswerable Question Detection in Clinical Data

ACL ARR 2024 August Submission18 Authors

Abstract:

Recent advancements in large-scale language models (LLMs) have focused research on generating SQL queries from domain-specific questions, particularly in the medical domain. A key challenge is detecting and filtering unanswerable questions, with traditional methods relying on model uncertainty, but these often require extra resources and lack interpretability. We propose a lightweight model that predicts relevant database schemas to detect unanswerable questions, enhancing interpretability and addressing the imbalance in binary classification tasks. Furthermore, we found that LLM-generated schema descriptions can significantly enhance the prediction accuracy. Our method provides a resource-efficient solution for unanswerable question detection in domain-specific question answering systems.
The source code is available at https://anonymous.4open.science/r/sqd2024

Paper Type: Short

Research Area: NLP Applications

Research Area Keywords: clinical NLP

Languages Studied: English

Submission Number: 18