TN-AutoRCA: Benchmark Construction and Agentic Framework for Self-Improving Alarm-Based Root Cause Analysis in Telecommunication Networks

AIMgroup, QianJin Yu

Keywords: Alarm-Based Root Cause Analysis; LLMs; Agentic Framework; Telecommunication Networks

TL;DR: This paper introduces TN-RCA, a new public benchmark for root cause analysis in telecommunication networks, and presents a novel agentic framework called Auto-RCA that dramatically improves an LLM's F1-score on this task from 58.99% to 91.79%.

Abstract:

Root Cause Analysis (RCA) in Telecommunication Networks is a critical task, yet it presents a formidable challenge for Artificial Intelligence (AI) due to its complex, graph-based reasoning requirements and the scarcity of realistic benchmarks. To catalyze research in this domain, We herein present TN-RCA, an inaugural real-world, publicly accessible benchmark for root cause analysis (RCA)
of telecommunication network alarms, comprising 530 fault scenarios constructed from expert-validated Knowledge Graphs(KGs). Our evaluation reveals that even state-of-the-art Large Language Models (LLMs) perform poorly on this task, with the best LLMs achieving an F1-score below 70%, highlighting its significant difficulty.To address this challenge, we then propose Auto-RCA, a novel agentic
system that produces the core code to analysis root cause through the automatically iterative refinement. The core innovation of Auto-RCA lies beyond simple self-correction; it employs an iterative ”evaluate-analyze-repair” loop that systematically identifies common patterns across all failure cases to generate contrastive feedback. This feedback guides the LLM to fix systemic logical flaws rather than
isolated errors. Experiments show that this agentic framework dramatically boosts the performance of root cause analysis in telecommunication networks, raising the final F1-score on TN-RCA from 58.99% (achieved by Gemini-2.5-Pro directly) to 91.79%. Meanwhile, Auto-RCA achieves 50% accuracy on the OpenRCA benchmark and demonstrates its generalization. This work not only
contributes a crucial benchmark to the community but also demonstrates that autonomous, self-optimizing agentic architecture is a powerful paradigm for solvingcomplex, domain-specific reasoning problems.

Primary Area: datasets and benchmarks

Submission Number: 7496