# WildGuard Benchmark Report (Prompt Harmfulness)

**Suite Run ID:** wildguard_wildguard_20250605_111029
**System/Model:** WildGuard Model: `allenai/wildguard` (Device: cuda)
**Timestamp Start:** 2025-06-05T11:10:29.439525
**Timestamp End:** 2025-06-05T11:20:01.903880
**Duration:** 572.46 seconds
**Input CSV Files:** [ANONYMIZED_PATH]/fortress_defense/data/05_stitched/ailuminate_english.csv
**Output Results File:** `wildguard_wildguard_20250605_111029_suite_results.json` (in results_data folder)
**Output Report File:** `wildguard_wildguard_20250605_111029_report.md` (in reports folder)
**Task:** Harmful User Request Detection

## Overall Metrics

| Metric                      | Value      |
|-----------------------------|------------|
| Number of Samples           | 1200 |
| Accuracy                    | 0.7950 |
| **UNSAFE Class Metrics**    |            |
| Precision (Unsafe)          | 1.0000 |
| Recall (Unsafe)             | 0.7950 |
| F1-score (Unsafe)           | 0.8858 |
| False Positive Rate (Unsafe)| 0.0000 |
| False Negative Rate (Unsafe)| 0.2050 |

## Counts

| Category                    | Count      |
|-----------------------------|------------|
| True Safe                   | 0 |
| True Unsafe                 | 1200 |
| Predicted Safe              | 246 |
| Predicted Unsafe            | 954 |
| Prediction Errors (Model)   | 0 |

## Confusion Matrix (Unsafe as Positive)

|                   | Predicted SAFE | Predicted UNSAFE |
|-------------------|----------------|------------------|
| **Actual SAFE**   | 0 (TN)     | 0 (FP)       |
| **Actual UNSAFE** | 246 (FN)     | 954 (TP)       |

---

**Notes:**
- Accuracy is calculated on samples where the true label was 'SAFE' or 'UNSAFE'. Predictions of 'ERROR' (or 'AMBIGUOUS' if applicable) against these true labels are considered misclassifications for accuracy.
- Precision, Recall, F1, FPR, FNR for the 'UNSAFE' class are calculated based on predictions that were either 'SAFE' or 'UNSAFE' and true labels that were also 'SAFE' or 'UNSAFE'.
- A detailed list of results for each prompt can be found in the data output file: `wildguard_wildguard_20250605_111029_suite_results.json`