REACT: Residual-Adaptive Contextual Tuning for Fast Model Adaptation in Cybersecurity
Keywords: Model Adaptation; Fine-Tuning; Threat Detection; Meta Learning; Hypernetwork
Abstract: Cybersecurity applications are challenged by constant distribution shifts due to the evolvement of services, users, and threats, degrading pretrained model performance. Fast adaptation is crucial for maintaining reliable security measures. Existing works primarily focus on pretraining models that can quickly adapt to new distributions, yet their fine-tuning relies on a rudimentary strategy that treats each shift independently. In this paper, we introduce REACT, a novel adaptation framework via Residual-Adaptive Contextual Tuning, working for sparsely-labeled and imbalanced cybersecurity data. REACT decomposes the weights of a neural network into two complementary components: meta weights, a shared foundation of general knowledge, and adaptive weights, tailored to specific shifts. A hypernetwork is employed to learn distribution patterns from few-shot data and relevant contexts and prime adaptive weights close to the optimal configuration, reducing fine-tuning effort. The meta weights and the hypernetwork are updated alternately to maximize generalization and adaptability. Extensive experiments across multiple datasets and neural networks demonstrate that REACT improves AUC by 14.85% compared to models without adaptation, outperforming the state-of-the-art.
Submission Number: 40
Loading