# Logging System

This directory contains log files generated by the prompt optimisation framework. The logging system provides comprehensive visibility into the framework's operation, helping with debugging, monitoring, and analysis.

## Log File Structure

Log files are organized as follows:

- **Experiment logs**: Each experiment run creates a log file in its output directory, named `experiment.log`.
- **General logs**: General framework logs are stored in this directory.

## Log Format

Log entries follow this format:

```
YYYY-MM-DD HH:MM:SS - module.name - LEVEL - Message
```

For example:
```
2025-06-30 15:45:23 - src.llm.openai_llm - INFO - Generated 150 tokens in 2.34s
```

## Log Levels

The framework uses the following log levels:

- **DEBUG**: Detailed information, typically useful only for diagnosing problems
- **INFO**: Confirmation that things are working as expected
- **WARNING**: Indication that something unexpected happened, but the application is still working
- **ERROR**: Due to a more serious problem, the application has not been able to perform a function
- **CRITICAL**: A serious error indicating that the application itself may be unable to continue running

## Controlling Log Verbosity

You can control log verbosity using command-line arguments:

```bash
# Enable verbose (DEBUG) logging
python examples/tamper_detection_experiment.py --verbose

# Enable quiet (WARNING) logging
python examples/tamper_detection_experiment.py --quiet

# Log to a specific file
python examples/tamper_detection_experiment.py --log-file custom_log.log
```

## Programmatic Logging

To add logging to your own code that uses this framework:

```python
from src.utils.logging_utils import get_logger

# Get a logger for your module
logger = get_logger(__name__)

# Log at different levels
logger.debug("Detailed information")
logger.info("Something happened")
logger.warning("Something unexpected happened")
logger.error("Something went wrong")
logger.critical("Something very bad happened")
```

## Log Analysis

Log files can be analyzed using standard tools like `grep`, `awk`, or specialized log analysis tools. For example:

```bash
# Find all error messages
grep "ERROR" logs/*.log

# Count occurrences of different log levels
grep -o " - INFO - " logs/*.log | wc -l
grep -o " - ERROR - " logs/*.log | wc -l

# Extract all OpenAI API calls
grep "Sending request to OpenAI API" logs/*.log
```

## Best Practices

1. Use appropriate log levels based on the importance of the information
2. Include relevant context in log messages
3. Log the start and end of important operations
4. Include timing information for performance-sensitive operations
5. Log input parameters and output results for key functions
6. Use structured logging for machine-readable logs when appropriate