T-FIX: Text-Based Explanations with Features Interpretable to eXperts

Shreya Havaldar, Helen Jin, Chaehyeon Kim, Anton Xue, Weiqiu You, Gary E. Weissman, Rajat Deo, Sameed Ahmed M. Khatana, Helen Qu, Marco Gatti, Daniel A Hashimoto, Amin Madani, Masao Sako, Bhuvnesh Jain, Lyle Ungar, Eric Wong

Keywords: interpretability, explainability, evaluation, domain-grounding, domain applications

Abstract:

As LLMs are deployed in knowledge‑intensive settings (e.g., surgery, astronomy, therapy), users expect not just answers, but also meaningful explanations for those answers.
In these settings, users are often domain experts (e.g., doctors, astrophysicists, psychologists) who require confidence that a model’s explanation reflects expert-level reasoning.
However, current evaluation schemes primarily emphasize plausibility or internal faithfulness of the explanation, often neglecting whether the content of the explanation truly aligns with expert intuition.
We formalize \textit{expert alignment} as a criterion for evaluating explanations with T‑FIX, a benchmark spanning seven knowledge-intensive domains. T-FIX includes datasets and novel alignment metrics developed in collaboration with domain experts, so an LLM’s explanations can be scored directly against expert judgment.

Submission Number: 104