A Meta-Analysis of Machine Learning Security Research: Attack-Defense Dynamics, Technical Evolution, and Cross-Concept Patterns

Abstract:

Machine learning (ML) security research has grown rapidly, yet systematic understanding of its technical evolution, attack-defense dynamics, and cross-concept patterns remains limited.
This study presents a systematic meta-analysis of 1,591 security papers spanning six security topics and five ML concepts, released between January 1, 2018, and June 30, 2024.
Beyond analyzing research trends, we quantify the attack-defense imbalance across all concept-topic combinations, revealing that defense research significantly lags behind attack research in emerging areas such as LLM jailbreaks (defense ratio = 0.30) and text-to-image membership inference (0.20).
Using LLM-assisted annotation of all paper abstracts, we identify 32 distinct technique families and trace their evolution over time, finding that attack techniques such as backdoor injection and adversarial perturbation first appeared in federated learning and graph neural networks before being adopted in LLM and text-to-image model security research.
We further identify 17 technique families shared across multiple ML concepts, with six spanning all five concepts studied.
Additionally, we examine factors associated with academic influence, finding that ML concepts, security topics, author count, regions, collaboration patterns, and publication status are all statistically significantly associated with citation density.
Our findings highlight critical defense gaps, map the technical landscape of ML security, and suggest concrete directions for future research.

Submission Type: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Fernando_Perez-Cruz1

Submission Number: 8330