 Container ai-scientist-python-workspace  Created
 Container ai-scientist-latex-compiler-1  Running
 Container ai-scientist-semantic-scholar-mcp-1  Running
Attaching to latex-compiler-1, ai-scientist-python-workspace, semantic-scholar-mcp-1
ai-scientist-python-workspace  | 2025-09-12 14:31:12,537 - WARNING - sklearn not available. Some baseline methods will be disabled.
ai-scientist-python-workspace  | 2025-09-12 14:31:12,538 - WARNING - sklearn not available. Using custom metric implementations.
ai-scientist-python-workspace  | 2025-09-12 14:31:12,703 - INFO - ================================================================================
ai-scientist-python-workspace  | 2025-09-12 14:31:12,704 - INFO - PHISHING EMAIL DETECTION EXPERIMENT
ai-scientist-python-workspace  | 2025-09-12 14:31:12,704 - INFO - ================================================================================
ai-scientist-python-workspace  | 2025-09-12 14:31:12,704 - INFO - 
ai-scientist-python-workspace  | [Step 1/5] Loading and preparing datasets...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,704 - INFO - Loading sample phishing dataset...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,704 - INFO - Generating synthetic phishing samples...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,706 - INFO - Dataset statistics:
ai-scientist-python-workspace  | 2025-09-12 14:31:12,706 - INFO -   Training samples: 700
ai-scientist-python-workspace  | 2025-09-12 14:31:12,706 - INFO -   Validation samples: 150
ai-scientist-python-workspace  | 2025-09-12 14:31:12,706 - INFO -   Test samples: 150
ai-scientist-python-workspace  | 2025-09-12 14:31:12,706 - INFO - 
ai-scientist-python-workspace  | [Step 2/5] Initializing baseline methods...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,706 - WARNING - TF-IDF + SVM not available due to missing sklearn
ai-scientist-python-workspace  | 2025-09-12 14:31:12,706 - INFO - 
ai-scientist-python-workspace  | [Step 3/5] Initializing hybrid LLM + rule-based detector...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,707 - WARNING - Could not connect to Ollama: [Errno 2] No such file or directory: 'ollama'. Using rule-based only.
ai-scientist-python-workspace  | 2025-09-12 14:31:12,707 - INFO - 
ai-scientist-python-workspace  | [Step 4/5] Running evaluations...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,707 - INFO - 
ai-scientist-python-workspace  | Evaluating baseline methods:
ai-scientist-python-workspace  | 2025-09-12 14:31:12,707 - INFO -   Testing Rule-based...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     Accuracy: 0.980
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     Precision: 1.000
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     Recall: 0.959
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     F1-Score: 0.979
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     Time: 0.00s
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -   Testing TF-IDF + SVM...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     Accuracy: 0.507
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     Precision: 0.500
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     Recall: 0.500
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     F1-Score: 0.500
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -     Time: 0.00s
ai-scientist-python-workspace  | 2025-09-12 14:31:12,708 - INFO -   Testing Regex Pattern...
ai-scientist-python-workspace  | 2025-09-12 14:31:12,713 - INFO -     Accuracy: 0.933
ai-scientist-python-workspace  | 2025-09-12 14:31:12,713 - INFO -     Precision: 1.000
ai-scientist-python-workspace  | 2025-09-12 14:31:12,713 - INFO -     Recall: 0.865
ai-scientist-python-workspace  | 2025-09-12 14:31:12,713 - INFO -     F1-Score: 0.928
ai-scientist-python-workspace  | 2025-09-12 14:31:12,713 - INFO -     Time: 0.00s
ai-scientist-python-workspace  | 2025-09-12 14:31:12,713 - INFO - 
ai-scientist-python-workspace  | Evaluating hybrid LLM + rule-based method:
ai-scientist-python-workspace  | 2025-09-12 14:31:12,713 - INFO - Training hybrid detector...
ai-scientist-python-workspace  | 2025-09-12 14:31:13,126 - INFO - Optimized weights: {'llm_score': 0.1, 'rule_score': 0.1, 'url_analysis': 0.1, 'sender_analysis': 0.7000000000000001}
ai-scientist-python-workspace  | 2025-09-12 14:31:13,128 - INFO - Hybrid detector training complete
ai-scientist-python-workspace  | 2025-09-12 14:31:13,134 - INFO -   Accuracy: 0.993
ai-scientist-python-workspace  | 2025-09-12 14:31:13,134 - INFO -   Precision: 1.000
ai-scientist-python-workspace  | 2025-09-12 14:31:13,134 - INFO -   Recall: 0.986
ai-scientist-python-workspace  | 2025-09-12 14:31:13,134 - INFO -   F1-Score: 0.993
ai-scientist-python-workspace  | 2025-09-12 14:31:13,134 - INFO -   Time: 0.42s
ai-scientist-python-workspace  | 2025-09-12 14:31:13,134 - INFO - 
ai-scientist-python-workspace  | [Step 5/5] Generating results report and visualizations...
ai-scientist-python-workspace  | 2025-09-12 14:31:13,134 - INFO - Generating visualizations...
ai-scientist-python-workspace  | 2025-09-12 14:31:13,752 - INFO - Visualizations saved to results_20250912_143112
ai-scientist-python-workspace  | 2025-09-12 14:31:13,753 - INFO - 
ai-scientist-python-workspace  | ================================================================================
ai-scientist-python-workspace  | 2025-09-12 14:31:13,753 - INFO - EXPERIMENT COMPLETED SUCCESSFULLY
ai-scientist-python-workspace  | 2025-09-12 14:31:13,753 - INFO - Results saved to: results_20250912_143112
ai-scientist-python-workspace  | 2025-09-12 14:31:13,753 - INFO - ================================================================================
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | ================================================================================
ai-scientist-python-workspace  | FINAL RESULTS COMPARISON
ai-scientist-python-workspace  | ================================================================================
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | Method                    Accuracy     Precision    Recall       F1-Score    
ai-scientist-python-workspace  | --------------------------------------------------------------------------------
ai-scientist-python-workspace  | Hybrid LLM + Rules        0.993        1.000        0.986        0.993       
ai-scientist-python-workspace  | Rule-based                0.980        1.000        0.959        0.979       
ai-scientist-python-workspace  | Regex Pattern             0.933        1.000        0.865        0.928       
ai-scientist-python-workspace  | TF-IDF + SVM              0.507        0.500        0.500        0.500       
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | ================================================================================

[Kai-scientist-python-workspace exited with code 0
