DOCUMENT GENERATION BENCHMARK - DETAILED ANALYSIS REPORT
============================================================

Generated on: 2025-09-17 17:20:09

BASIC STATISTICS
------------------------------
Total queries processed: 40
Average user profile accuracy: 0.445
Average intent capture accuracy: 0.525
Average citation accuracy: 0.110
Average document quality score: 4.549
Overall average score: 1.146

ADVANCED METRICS
------------------------------
Score Statistics:
  user_profile_accuracy:
    Mean: 0.445 ± 0.185
    Median: 0.455
    Range: 0.153 - 0.795

  intent_capture_accuracy:
    Mean: 0.525 ± 0.174
    Median: 0.600
    Range: 0.200 - 0.800

  citation_accuracy:
    Mean: 0.110 ± 0.197
    Median: 0.000
    Range: 0.000 - 0.667

  document_quality_score:
    Mean: 4.549 ± 0.568
    Median: 4.700
    Range: 2.700 - 5.000

Correlations:
  profile_intent_correlation: -0.399
  intent_quality_correlation: -0.100
  citation_quality_correlation: 0.188
  profile_quality_correlation: 0.314

PERFORMANCE BY DOCUMENT TYPE
------------------------------
email: 1.135 (n=15)
status_report: 1.192 (n=19)
faq: 1.028 (n=6)

PERFORMANCE BY USER ROLE
------------------------------
DevOps Engineer: 1.097 (n=1)
Business Analyst: 1.154 (n=7)
IT Systems Lead: 1.279 (n=2)
Project Manager: 1.161 (n=20)
Product Manager: 1.004 (n=2)
Data Analyst: 1.123 (n=4)
UX Designer: 1.094 (n=2)
Software Engineer: 1.100 (n=2)

CITATION ANALYSIS
------------------------------
Documents with citations: 40/40
Average citations per document: 22.73

Most cited messages:
  Msg_203: 58 citations
  Msg_390: 33 citations
  Msg_477: 32 citations
  Msg_709: 25 citations
  Msg_1081: 20 citations

QUALITY DIMENSIONS ANALYSIS
------------------------------
citation_quality: 4.350 ± 0.654
