 Container ai-scientist-latex-compiler-1  Running
 Container ai-scientist-semantic-scholar-mcp-1  Running
 Container ai-scientist-python-workspace  Created
Attaching to latex-compiler-1, ai-scientist-python-workspace, semantic-scholar-mcp-1
ai-scientist-python-workspace  | 2025-09-12 09:06:04,836 - WARNING - sklearn not available. Some baseline methods will be disabled.
ai-scientist-python-workspace  | 2025-09-12 09:06:04,842 - WARNING - sklearn not available. Using custom metric implementations.
ai-scientist-python-workspace  | 2025-09-12 09:06:05,514 - INFO - generated new fontManager
ai-scientist-python-workspace  | 2025-09-12 09:06:06,063 - INFO - ================================================================================
ai-scientist-python-workspace  | 2025-09-12 09:06:06,063 - INFO - PHISHING EMAIL DETECTION EXPERIMENT
ai-scientist-python-workspace  | 2025-09-12 09:06:06,063 - INFO - ================================================================================
ai-scientist-python-workspace  | 2025-09-12 09:06:06,063 - INFO - 
ai-scientist-python-workspace  | [Step 1/5] Loading and preparing datasets...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,064 - INFO - Loading sample phishing dataset...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,064 - INFO - Generating synthetic phishing samples...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,067 - INFO - Dataset statistics:
ai-scientist-python-workspace  | 2025-09-12 09:06:06,067 - INFO -   Training samples: 700
ai-scientist-python-workspace  | 2025-09-12 09:06:06,067 - INFO -   Validation samples: 150
ai-scientist-python-workspace  | 2025-09-12 09:06:06,067 - INFO -   Test samples: 150
ai-scientist-python-workspace  | 2025-09-12 09:06:06,067 - INFO - 
ai-scientist-python-workspace  | [Step 2/5] Initializing baseline methods...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,067 - WARNING - TF-IDF + SVM not available due to missing sklearn
ai-scientist-python-workspace  | 2025-09-12 09:06:06,068 - INFO - 
ai-scientist-python-workspace  | [Step 3/5] Initializing hybrid LLM + rule-based detector...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,068 - WARNING - Could not connect to Ollama: [Errno 2] No such file or directory: 'ollama'. Using rule-based only.
ai-scientist-python-workspace  | 2025-09-12 09:06:06,068 - INFO - 
ai-scientist-python-workspace  | [Step 4/5] Running evaluations...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,068 - INFO - 
ai-scientist-python-workspace  | Evaluating baseline methods:
ai-scientist-python-workspace  | 2025-09-12 09:06:06,068 - INFO -   Testing Rule-based...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,072 - INFO -     Accuracy: 0.993
ai-scientist-python-workspace  | 2025-09-12 09:06:06,072 - INFO -     Precision: 1.000
ai-scientist-python-workspace  | 2025-09-12 09:06:06,072 - INFO -     Recall: 0.987
ai-scientist-python-workspace  | 2025-09-12 09:06:06,072 - INFO -     F1-Score: 0.993
ai-scientist-python-workspace  | 2025-09-12 09:06:06,072 - INFO -     Time: 0.00s
ai-scientist-python-workspace  | 2025-09-12 09:06:06,072 - INFO -   Testing TF-IDF + SVM...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,074 - INFO -     Accuracy: 0.560
ai-scientist-python-workspace  | 2025-09-12 09:06:06,074 - INFO -     Precision: 0.585
ai-scientist-python-workspace  | 2025-09-12 09:06:06,074 - INFO -     Recall: 0.494
ai-scientist-python-workspace  | 2025-09-12 09:06:06,074 - INFO -     F1-Score: 0.535
ai-scientist-python-workspace  | 2025-09-12 09:06:06,074 - INFO -     Time: 0.00s
ai-scientist-python-workspace  | 2025-09-12 09:06:06,074 - INFO -   Testing Regex Pattern...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,082 - INFO -     Accuracy: 0.947
ai-scientist-python-workspace  | 2025-09-12 09:06:06,082 - INFO -     Precision: 1.000
ai-scientist-python-workspace  | 2025-09-12 09:06:06,082 - INFO -     Recall: 0.896
ai-scientist-python-workspace  | 2025-09-12 09:06:06,082 - INFO -     F1-Score: 0.945
ai-scientist-python-workspace  | 2025-09-12 09:06:06,082 - INFO -     Time: 0.01s
ai-scientist-python-workspace  | 2025-09-12 09:06:06,082 - INFO - 
ai-scientist-python-workspace  | Evaluating hybrid LLM + rule-based method:
ai-scientist-python-workspace  | 2025-09-12 09:06:06,082 - INFO - Training hybrid detector...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,684 - INFO - Optimized weights: {'llm_score': 0.1, 'rule_score': 0.1, 'url_analysis': 0.1, 'sender_analysis': 0.7000000000000001}
ai-scientist-python-workspace  | 2025-09-12 09:06:06,689 - INFO - Hybrid detector training complete
ai-scientist-python-workspace  | 2025-09-12 09:06:06,698 - INFO -   Accuracy: 1.000
ai-scientist-python-workspace  | 2025-09-12 09:06:06,698 - INFO -   Precision: 1.000
ai-scientist-python-workspace  | 2025-09-12 09:06:06,698 - INFO -   Recall: 1.000
ai-scientist-python-workspace  | 2025-09-12 09:06:06,698 - INFO -   F1-Score: 1.000
ai-scientist-python-workspace  | 2025-09-12 09:06:06,698 - INFO -   Time: 0.62s
ai-scientist-python-workspace  | 2025-09-12 09:06:06,698 - INFO - 
ai-scientist-python-workspace  | [Step 5/5] Generating results report and visualizations...
ai-scientist-python-workspace  | 2025-09-12 09:06:06,698 - INFO - Generating visualizations...
ai-scientist-python-workspace  | 2025-09-12 09:06:07,758 - INFO - Visualizations saved to results_20250912_090606
ai-scientist-python-workspace  | 2025-09-12 09:06:07,760 - INFO - 
ai-scientist-python-workspace  | ================================================================================
ai-scientist-python-workspace  | 2025-09-12 09:06:07,760 - INFO - EXPERIMENT COMPLETED SUCCESSFULLY
ai-scientist-python-workspace  | 2025-09-12 09:06:07,760 - INFO - Results saved to: results_20250912_090606
ai-scientist-python-workspace  | 2025-09-12 09:06:07,760 - INFO - ================================================================================
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | ================================================================================
ai-scientist-python-workspace  | FINAL RESULTS COMPARISON
ai-scientist-python-workspace  | ================================================================================
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | Method                    Accuracy     Precision    Recall       F1-Score    
ai-scientist-python-workspace  | --------------------------------------------------------------------------------
ai-scientist-python-workspace  | Hybrid LLM + Rules        1.000        1.000        1.000        1.000       
ai-scientist-python-workspace  | Rule-based                0.993        1.000        0.987        0.993       
ai-scientist-python-workspace  | Regex Pattern             0.947        1.000        0.896        0.945       
ai-scientist-python-workspace  | TF-IDF + SVM              0.560        0.585        0.494        0.535       
ai-scientist-python-workspace  | 
ai-scientist-python-workspace  | ================================================================================

[Kai-scientist-python-workspace exited with code 0
