 Container ai-scientist-python-workspace  Created
 Container ai-scientist-latex-compiler-1  Running
 Container ai-scientist-semantic-scholar-mcp-1  Running
Attaching to latex-compiler-1, ai-scientist-python-workspace, semantic-scholar-mcp-1
ai-scientist-python-workspace  | 2025-09-16 04:55:11,713 - WARNING - sklearn not available. Some baseline methods will be disabled.
ai-scientist-python-workspace  | 2025-09-16 04:55:11,714 - WARNING - sklearn not available. Using custom metric implementations.
ai-scientist-python-workspace  | 2025-09-16 04:55:12,227 - INFO - ================================================================================
ai-scientist-python-workspace  | 2025-09-16 04:55:12,227 - INFO - PHISHING DETECTION - ACADEMIC METHODS COMPARISON
ai-scientist-python-workspace  | 2025-09-16 04:55:12,227 - INFO - ================================================================================
ai-scientist-python-workspace  | 2025-09-16 04:55:12,227 - INFO - 
ai-scientist-python-workspace  | [Step 1/4] Loading and preparing datasets...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,227 - INFO - Downloading and preparing real phishing datasets...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,227 - INFO - Loading cached combined dataset...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,228 - INFO - Loaded 1002 emails from cache
ai-scientist-python-workspace  | 2025-09-16 04:55:12,229 - INFO - Dataset statistics:
ai-scientist-python-workspace  | 2025-09-16 04:55:12,229 - INFO -   Training samples: 701
ai-scientist-python-workspace  | 2025-09-16 04:55:12,229 - INFO -   Validation samples: 150
ai-scientist-python-workspace  | 2025-09-16 04:55:12,229 - INFO -   Test samples: 151
ai-scientist-python-workspace  | 2025-09-16 04:55:12,229 - INFO - 
ai-scientist-python-workspace  | [Step 2/4] Initializing detection methods...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,229 - WARNING - TF-IDF + SVM not available due to missing sklearn
ai-scientist-python-workspace  | 2025-09-16 04:55:12,229 - INFO -   - PhishIntention adapter (USENIX 2022)...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,229 - INFO -   - CNN-BiGRU detector (Sensors 2024)...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,237 - INFO - CNN-BiGRU model initialized on cpu
ai-scientist-python-workspace  | 2025-09-16 04:55:12,237 - INFO -   - Feature Ensemble detector (uOttawa 2023)...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,237 - INFO -   - Hybrid LLM-Regex detector (Ours)...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,238 - INFO - 
ai-scientist-python-workspace  | [Step 3/4] Running evaluations...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,238 - INFO - 
ai-scientist-python-workspace  | Evaluating all detection methods:
ai-scientist-python-workspace  | 2025-09-16 04:55:12,238 - INFO - 
ai-scientist-python-workspace  |   Testing Rule-based Baseline...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,239 - INFO -     Accuracy: 0.000
ai-scientist-python-workspace  | 2025-09-16 04:55:12,239 - INFO -     Precision: 0.000
ai-scientist-python-workspace  | 2025-09-16 04:55:12,239 - INFO -     Recall: 0.000
ai-scientist-python-workspace  | 2025-09-16 04:55:12,239 - INFO -     F1-Score: 0.000
ai-scientist-python-workspace  | 2025-09-16 04:55:12,239 - INFO -     Time: 0.00s
ai-scientist-python-workspace  | 2025-09-16 04:55:12,239 - INFO - 
ai-scientist-python-workspace  |   Testing Regex Pattern Baseline...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,240 - INFO -     Accuracy: 0.000
ai-scientist-python-workspace  | 2025-09-16 04:55:12,240 - INFO -     Precision: 0.000
ai-scientist-python-workspace  | 2025-09-16 04:55:12,240 - INFO -     Recall: 0.000
ai-scientist-python-workspace  | 2025-09-16 04:55:12,240 - INFO -     F1-Score: 0.000
ai-scientist-python-workspace  | 2025-09-16 04:55:12,240 - INFO -     Time: 0.00s
ai-scientist-python-workspace  | 2025-09-16 04:55:12,240 - INFO - 
ai-scientist-python-workspace  |   Testing PhishIntention (USENIX'22)...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,240 - INFO - PhishIntention adapter initialized (no training required)
ai-scientist-python-workspace  | 2025-09-16 04:55:12,241 - INFO - Validation accuracy on sample: 0.45
ai-scientist-python-workspace  | 2025-09-16 04:55:12,241 - ERROR -     Error evaluating PhishIntention (USENIX'22): 'list' object has no attribute 'get'
ai-scientist-python-workspace  | 2025-09-16 04:55:12,241 - INFO - 
ai-scientist-python-workspace  |   Testing CNN-BiGRU (Sensors'24)...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,241 - INFO - Training CNN-BiGRU phishing detector...
ai-scientist-python-workspace  | 2025-09-16 04:55:12,243 - INFO - Vocabulary built with 331 words
ai-scientist-python-workspace  | 2025-09-16 04:55:13,150 - INFO - Epoch 1/5: Loss=0.6973, Accuracy=0.475
ai-scientist-python-workspace  | 2025-09-16 04:55:13,712 - INFO - Epoch 2/5: Loss=0.6952, Accuracy=0.490
ai-scientist-python-workspace  | 2025-09-16 04:55:14,320 - INFO - Epoch 3/5: Loss=0.6949, Accuracy=0.485
ai-scientist-python-workspace  | 2025-09-16 04:55:14,914 - INFO - Epoch 4/5: Loss=0.6898, Accuracy=0.560
ai-scientist-python-workspace  | 2025-09-16 04:55:15,509 - INFO - Epoch 5/5: Loss=0.6907, Accuracy=0.460
ai-scientist-python-workspace  | 2025-09-16 04:55:15,555 - INFO - Validation accuracy: 0.520
ai-scientist-python-workspace  | 2025-09-16 04:55:15,557 - ERROR -     Error evaluating CNN-BiGRU (Sensors'24): 'list' object has no attribute 'get'
ai-scientist-python-workspace  | 2025-09-16 04:55:15,557 - INFO - 
ai-scientist-python-workspace  |   Testing Feature Ensemble (uOttawa'23)...
ai-scientist-python-workspace  | 2025-09-16 04:55:15,557 - INFO - Training Feature-based Ensemble Detector...
ai-scientist-python-workspace  | 2025-09-16 04:55:15,568 - ERROR -     Error evaluating Feature Ensemble (uOttawa'23): No module named 'sklearn'
ai-scientist-python-workspace  | 2025-09-16 04:55:15,568 - INFO - 
ai-scientist-python-workspace  |   Testing Hybrid LLM-Regex (Ours)...
ai-scientist-python-workspace  | 2025-09-16 04:55:15,568 - INFO - Training Hybrid LLM-Regex Detector...
ai-scientist-python-workspace  | 2025-09-16 04:55:15,569 - INFO - Optimized regex threshold: 2.00
ai-scientist-python-workspace  | 2025-09-16 04:55:15,569 - ERROR -     Error evaluating Hybrid LLM-Regex (Ours): expected string or bytes-like object, got 'dict'
ai-scientist-python-workspace  | 2025-09-16 04:55:15,569 - INFO - 
ai-scientist-python-workspace  | [Step 4/4] Generating results report and visualizations...
ai-scientist-python-workspace  | 2025-09-16 04:55:15,569 - INFO - Generating visualizations...
ai-scientist-python-workspace  | 2025-09-16 04:55:16,196 - INFO - Visualizations saved to results_20250916_045512
ai-scientist-python-workspace  | /app/workspace/main.py:221: RuntimeWarning: invalid value encountered in scalar divide
ai-scientist-python-workspace  |   improvement = ((academic_avg - baseline_avg) / baseline_avg) * 100
ai-scientist-python-workspace  | 2025-09-16 04:55:16,197 - INFO - 
ai-scientist-python-workspace  | ================================================================================
ai-scientist-python-workspace  | 2025-09-16 04:55:16,197 - INFO - EXPERIMENT COMPLETED SUCCESSFULLY
ai-scientist-python-workspace  | 2025-09-16 04:55:16,197 - INFO - Results saved to: results_20250916_045512
ai-scientist-python-workspace  | 2025-09-16 04:55:16,197 - INFO - ================================================================================
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | ====================================================================================================
ai-scientist-python-workspace  | FINAL RESULTS COMPARISON - ACADEMIC METHODS
ai-scientist-python-workspace  | ====================================================================================================
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | Method                              Accuracy     Precision    Recall       F1-Score     Time(s)   
ai-scientist-python-workspace  | ----------------------------------------------------------------------------------------------------
ai-scientist-python-workspace  | Rule-based Baseline                 0.000        0.000        0.000        0.000        0.00      
ai-scientist-python-workspace  | Regex Pattern Baseline              0.000        0.000        0.000        0.000        0.00      
ai-scientist-python-workspace  | PhishIntention (USENIX'22)          0.000        0.000        0.000        0.000        0.00      
ai-scientist-python-workspace  | CNN-BiGRU (Sensors'24)              0.000        0.000        0.000        0.000        0.00      
ai-scientist-python-workspace  | Feature Ensemble (uOttawa'23)       0.000        0.000        0.000        0.000        0.00      
ai-scientist-python-workspace  | Hybrid LLM-Regex (Ours)             0.000        0.000        0.000        0.000        0.00      
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | ====================================================================================================

[Kai-scientist-python-workspace exited with code 0
