HyperTTC: Hypergraph-Empowered Tactic-Specific Traffic Clustering for Atomized APT Detection

Wenhui Du, Yuanhang He, Gaolei Li, Xiao Yang, Jianhua Li, Ge Ren, Kai Zhou

Abstract: Cybersecurity professionals often encounter a significant semantic gap between low-level traffic audits and high-level system behaviors, which hinders effective tactic recognition of advanced persistent threats (APTs). Existing provenance graph-based traffic clustering methods uses binary relation to detect attack actions, which can not adequately capture complex interactions among multiple entities. To identify attack tactics from fragmented traffic audits on multiple entities, in this paper we propose a novel hypergraph-empowered tactic-specific traffic clustering (HyperTTC) scheme, which leverages the transformative potential of hypergraphs to aggregate entities that carry the same APT attack tactic together. Different from existing methods, HyperTTC is capable of achieving atomized APT detection, where each attack tactic can be identified with the combination of multi-dimension relations. By constructing a hypergraph structure of fragmented traffic audits, HyperTTC provides an exhaustive representation of APT behaviors, thereby enhancing detection precision and bolstering resilience against sophisticated attack strategies. Extensive experiments on real-world datasets validate the effectiveness of HyperTTC for the F1 score is 12.5% higher than the state of the art method.