# Model Performance Leaderboards

This directory contains comprehensive visualizations of model performance across different datasets, broken down into focused, easy-to-read charts.

## Generated Files

### Cisco Dataset Analysis
- **`cisco_performance_ranking.png`** - Clean horizontal bar chart ranking models by F1 score
- **`cisco_metrics_comparison.png`** - Individual bar charts for Accuracy, Precision, Recall, and F1
- **`cisco_performance_correlations.png`** - Scatter plots showing metric correlations
- **`cisco_training_time_analysis.png`** - Training time comparison and performance vs time analysis

### NSL-KDD Dataset Analysis
- **`nsl_kdd_performance_ranking.png`** - Clean horizontal bar chart ranking models by F1 score
- **`nsl_kdd_metrics_comparison.png`** - Individual bar charts for Accuracy, Precision, Recall, and F1
- **`nsl_kdd_performance_correlations.png`** - Scatter plots showing metric correlations
- **`nsl_kdd_training_time_analysis.png`** - Training time comparison and performance vs time analysis

### Multi-class Analysis (NSL-KDD)
#### Per-Metric Heatmaps
- **`nsl_kdd_class_accuracy_heatmap.png`** - Accuracy heatmap across all models and attack classes
- **`nsl_kdd_class_precision_heatmap.png`** - Precision heatmap across all models and attack classes
- **`nsl_kdd_class_recall_heatmap.png`** - Recall heatmap across all models and attack classes
- **`nsl_kdd_class_f1_heatmap.png`** - F1 score heatmap across all models and attack classes

#### Per-Class Rankings
- **`nsl_kdd_class_dos_ranking.png`** - Model performance ranking for DoS attacks
- **`nsl_kdd_class_normal_ranking.png`** - Model performance ranking for Normal traffic
- **`nsl_kdd_class_probe_ranking.png`** - Model performance ranking for Probe attacks
- **`nsl_kdd_class_r2l_ranking.png`** - Model performance ranking for R2L attacks
- **`nsl_kdd_class_u2r_ranking.png`** - Model performance ranking for U2R attacks

#### Per-Model Class Analysis
- **`nsl_kdd_model_[model]_class_comparison.png`** - How each model performs across different attack classes

### Cross-Dataset Summary
- **`summary_comparison.png`** - Cross-dataset performance comparison and model rankings

### Raw Data
- **`all_results.csv`** - Raw performance metrics for all models and datasets

## How to Regenerate

Run the visualization script from the project root:

```bash
python -m gatv2_ns3_ids.scripts.visualize_results
```

The script will:
1. Load all `results.json` files from the `outputs/` directory
2. Extract performance metrics for each model-dataset combination
3. Generate leaderboard visualizations
4. Handle multi-class metrics with per-label information
5. Save all visualizations to the `leaderboards/` folder

## Metrics Included

- **Accuracy** - Overall classification accuracy
- **Precision** - Weighted average precision
- **Recall** - Weighted average recall  
- **F1 Score** - Weighted average F1 score
- **AUC** - Area under the ROC curve
- **Training Time** - Total training time in seconds
- **Per-class metrics** - Individual accuracy, precision, recall, and F1 for each attack type (NSL-KDD only)

## Datasets

- **CISCO** - Network intrusion detection on Cisco network data
- **NSL-KDD** - Multi-class intrusion detection with attack types: Normal, DoS, Probe, R2L, U2R
