United We Log, Divided We Identify: A Decentralized Approach for Automated Log Analysis

Elnaz Rabieinejad, Ali Dehghantanha, Fattane Zarrinkalam, Jeff Schwartzentruber

Abstract: Log analysis plays a crucial role in ensuring endpoint security, but it is hindered by the lack of sufficient contextual information in logs, making it challenging to interpret endpoint behaviors accurately and significantly increasing the effort required to detect malicious activities. Existing approaches, which are predominantly manual or semi-automated, struggle to manage this workload and frequently fail to deliver the contextual insights. Moreover, centralizing sensitive log data on one analysis server compounds security risks, amplifying breach potential and creating a dangerous single point of failure. These vulnerabilities demand robust, context-aware solutions; to meet that need, this paper presents Federated LogTracer, a novel system that overcomes the shortcomings of current log-analysis methods. Federated LogTracer offers an automated and decentralized strategy for extracting contextual data, identifying malicious logs, and reducing workload, while improving the privacy and compliance of data for log owners. By leveraging an advanced parse graph generation technique and federated learning, the system efficiently extracts meaningful contextual data with minimal effort. Additionally, it ensures sensitive log information remains localized, safeguarding privacy. To mitigate the risks associated with centralization and the resulting single point of failure in existing solutions, Federated LogTracer incorporates a decentralized federated learning framework, providing a robust and secure approach to log analysis. Federated LogTracer achieves outstanding performance on the DARPA Transparent Computing dataset, with detection metrics exceeding 99% across various endpoint log sets. By shrinking complexity from linear O(n) to sub-linear \(O((t/m)+p log(n/m))\) as logs and clients grow, it cuts manual effort. Its decentralized design retains data locally, slashing privacy exposure and transmission overhead, typical in centralized systems architectures.

External IDs:dblp:conf/acns/RabieinejadDZS25