XBRL Agent: Leveraging Large Language Models for Financial Report Analysis

Shijie Han, Haoqiang Kang, Bo Jin, Xiao-Yang Liu, Steve Y. Yang

Abstract: eXtensible Business Reporting Language (XBRL) has attained the status of the global de facto standard for business reporting. However, its complexity poses significant barriers to interpretation and accessibility. In this paper, we present the first evaluation of large language models’ (LLMs) performance in analyzing XBRL reports. Our study identifies LLMs’ limitations in the comprehension of financial domain knowledge and mathematical calculation in the context of XBRL reports. To address these issues, we propose enhancement methods using external tools under the agent framework, referred to as XBRL-Agent, which invokes retrievers and calculators. Extensive experiments on two tasks - the Domain Query Task (which involved testing 500 XBRL term explanations and 50 domain questions) and the Numeric Type Query Task (tested 1,000 financial math tests and 50 numeric queries) - demonstrate substantial performance improvements, with accuracy increasing by up to 17% for the domain task and 42% for the numeric type task. This work not only explores the potential of LLMs for analyzing XBRL reports but also augments the reliability and robustness of such analysis, although there is still much room for improvement in mathematical calculations.