TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law

ICLR 2026 Conference Submission21016 Authors

Keywords: Safety, Benchmark, Large language model

TL;DR: Benchmarking LLM safety in Finance, Medicine, and Law

Abstract: As large language models (LLMs) are increasingly deployed in high-risk domains such as law, finance, and medicine, systematically evaluating their domain-specific safety and compliance becomes critical. While prior work has largely focused on improving LLM performance in these domains, it has often neglected the evaluation of domain-specific safety risks. To bridge this gap, we first define domain-specific safety principles for LLMs based on the AMA Principles of Medical Ethics, the ABA Model Rules of Professional Conduct, and the CFA Institute Code of Ethics. Building on this foundation, we introduce Trident-Bench, a benchmark specifically targeting LLM safety in the legal, financial, and medical domains. We evaluated 19 general-purpose and domain-specialized models on Trident-Bench and show that it effectively reveals key safety gaps: strong generalist models can meet basic expectations, whereas domain-specialized models often fail. This highlights an urgent need for more robust safeguards in high-stakes domains. By introducing Trident-Bench, our work provides one of the first systematic resources for studying LLM safety in law and finance, and lays the groundwork for future research aimed at reducing the safety risks of deploying LLMs in professionally regulated fields. Code and benchmark will be released.

Primary Area: datasets and benchmarks

Submission Number: 21016