HyTek-P: Benchmark for Hybrid Reasoning over Real-World Tables under Privacy Constraints

ACL ARR 2026 January Submission10829 Authors

Keywords: Privacy, Natural Language Processing, Large Language Models, Reasoning Models.

Abstract: Large language models are increasingly deployed in enterprise and regulatory settings where reasoning must be performed over heterogeneous table–text data under strict privacy constraints. However, existing benchmarks for tabular or hybrid question answering largely assume clean, unredacted inputs and therefore fail to capture the systematic information loss introduced by real-world privacy practices such as masking, deletion, and generalization. This gap obscures how redaction fundamentally alters model reasoning behavior. We present HyTekP (Hybrid Text–Knowledge reasoning under Privacy Constraints), a benchmark designed to evaluate privacy-aware hybrid reasoning over semi-structured tables and unstructured text. HyTekP is constructed from three high-impact real-world domains i.e., consumer finance complaints, clinical records, and police reports, each paired with realistic redaction strategies and expert-validated queries. Tasks span five core analytic operation types, enabling fine-grained analysis of reasoning failures. We further introduce a diagnostic error taxonomy and redacted–unredacted comparisons to isolate privacy-induced degradation. HyTekP provides a rigorous testbed for developing and evaluating models robust to privacy-preserving data transformations.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: clinical NLP;multimodal applications;security/privacy;financial/business NLP;commonsense QA; biomedical QA;table QA;few-shot QA

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data resources, Data analysis

Languages Studied: Hindi, English

Submission Number: 10829