# LMTune Agent Documentation

## Overview

The `lmtune-agent` is an AI-powered system that generates Python scripts for MiniZinc constraint solvers (algorithms)selection. It analyzes problem instances and creates scripts that recommend optimal solvers (algorithms) based on instance characteristics.

## Agent Architecture

The agent operates as a multi-step workflow:

1. **Problem Analysis**: Reads and analyzes MiniZinc problem instances in JSON format
2. **Template Loading**: Loads universal MiniZinc tuning prompts and problem-specific schemas
3. **Script Generation**: Uses LLMs to generate Python analysis scripts
4. **Execution & Validation**: Runs generated scripts and validates output
5. **Archiving**: Stores successful scripts for future reference

## Tools Available to the Agent

The LangGraph ReAct agent has access to exactly 5 tools for managing and executing Python scripts:

### Script Editor Tools
1. **clear()**: Clear the current script content completely
2. **view(view_range=None)**: View the script content, optionally within a specific line range [start, end] (1-indexed)
3. **insert(insert_line, new_str)**: Insert text at a specific line position (0 for beginning, or after line N)
4. **check()**: Validate the Python code syntax and ensure it contains required `output_results()` call
5. **execute_script()**: Execute the Python script with timeout protection and return output

### What Happens When execute_script() is Called

When the ReAct agent calls `execute_script()`, here's the exact sequence:

1. **Validation**: First validates the script syntax using the `check()` logic
2. **File Loading**: Uses Python's `inspect` module to access the command line arguments (`self.args`) and loads the instance file from `{problem_folder}/{problem_type}/{instance_file}` 
   - Command line args stored when ScriptEditor was created
   - Tries to parse as JSON first
   - Falls back to plain text if JSON parsing fails
3. **Helper Module Creation**: Creates a temporary `lmtune_helpers.py` file containing:
   ```python
   _INPUT_DATA = {the_loaded_instance_data}
   
   def input_data():
       return _INPUT_DATA
   
   def output_results(result_dict):
       print(json.dumps(result_dict))
   ```
4. **Script File Creation**: Creates a temporary `.py` file with:
   - UV script header with dependencies (pydantic, networkx, numpy)
   - The agent's generated script content
5. **Subprocess Execution**: Runs `uv run temp_script.py` with:
   - Timeout protection (default 30 seconds)
   - PYTHONPATH set to find the helper module
   - Captures stdout/stderr
6. **Cleanup**: Deletes temporary files
7. **Return Result**: Returns the script output or formatted error message

### Built-in Helper Functions
The agent's scripts automatically have access to:
- **input_data()**: Get the loaded JSON problem instance data (from the command line arguments)
- **output_results(dict)**: Format and print results as JSON (required for all scripts)
- Standard Python libraries: json, networkx, numpy, pydantic

## Task Workflow

### Phase 1: Input Processing
1. **Load Problem Data**: Read JSON instance file containing problem parameters
2. **Load Schema**: Read `json-schema.md` to understand data structure
3. **Load MiniZinc Model**: Read `.mzn` file to understand constraint model
4. **Template Assembly**: Combine universal tuning prompt with problem-specific details

### Phase 2: Script Generation
1. **LLM Analysis**: Send assembled prompt to language model
2. **Iterative Code Generation**: LLM uses tools to build the script step by step:
   - **insert()**: Add code blocks (imports, functions, analysis logic)
   - **view()**: Review current script content
   - **check()**: Validate syntax and required function calls
3. **Script Requirements**: Generated scripts must call `output_results(dict)` at the end

### Phase 3: Execution & Output
1. **Script Execution**: Agent calls **execute_script()** tool with timeout protection
2. **Automatic Data Loading**: System loads the JSON instance file specified in command line arguments and makes it available via `input_data()` function
3. **Output Parsing**: Extract JSON results containing:
   - Instance characteristics (size, complexity metrics)
   - Recommended solver parameters
   - Parameter justifications
   - Performance impact estimates
4. **Result Validation**: Ensure output matches expected JSON schema

### Phase 4: Archiving
1. **Script Storage**: Save successful script as `lmtuner[timestamp].py`
2. **Result Display**: Present recommendations to user
3. **Error Handling**: Log failures and provide debugging information

## Input Requirements

### Problem Directory Structure
```
problems/[problem-name]/
├── [problem].mzn          # MiniZinc constraint model
├── json-schema.md         # JSON format documentation
├── example.dzn            # Example input (optional)
├── example.json           # Example JSON (optional)
└── *.json                 # Problem instances
```

### JSON Instance Format
Problem instances must be in JSON format with structure matching the schema defined in `json-schema.md`. Common fields include:
- Problem size parameters (e.g., `N`, `capacity`)
- Input data arrays (e.g., `distances`, `demands`, `weights`)
- Graph structures for network problems

## Output Format

The agent generates scripts that output JSON with this structure:

```json
{
  "instance_characteristics": {
    "size_metrics": "...",
    "complexity_measures": "...",
    "problem_specific_features": "..."
  },
  "recommended_parameters": {
    "parameter_name": "value",
    "another_parameter": "value"
  },
  "parameter_justifications": {
    "parameter_name": "reasoning for this value",
    "another_parameter": "reasoning for this value"
  },
  "estimated_performance_impact": {
    "expected_speedup_percent": "number",
    "notes": "additional performance notes"
  }
}
```

## Supported Problem Types

### Current Problems
- **VRP**: Vehicle Routing Problem with capacity constraints
- **Graph-Clear**: Vertex cover problems on graphs
- **CVRP**: Capacitated Vehicle Routing Problem variants

### Problem-Specific Analysis
Each problem type requires different analysis approaches:

- **Graph Problems**: Node/edge counts, density, degree distribution
- **Routing Problems**: Customer counts, capacity utilization, geographic distribution
- **Scheduling Problems**: Resource constraints, time horizons, dependency complexity

## Error Handling

The agent handles various error conditions through its tools:

1. **Syntax Errors**: **check()** tool validates Python syntax and provides detailed error messages with hints
2. **Execution Errors**: **execute_script()** tool catches and categorizes runtime errors:
   - ImportError, NameError, SyntaxError, IndexError, KeyError, TypeError, etc.
   - Provides specific guidance for each error type
3. **Timeout Errors**: Scripts terminated if they exceed the timeout limit (default: 30 seconds)
4. **Output Validation**: Scripts must call `output_results()` or they're considered invalid
5. **Missing Data**: Automatic loading of instance data through `input_data()` function

## Configuration

### Model Selection
- `--mc`: Specify LLM provider and model
- Supported: OpenAI, Anthropic, OpenRouter, Google, Local models

### Execution Control
- `--timeout`: Script execution timeout (default: 30 seconds)
- `--problem-folder`: Custom problem directory location

### Problem Selection
- `--problem`: Problem type subdirectory name
- `--instance`: Specific JSON instance file

## Best Practices

### For Users
1. Ensure JSON instances match the schema exactly
2. Use descriptive instance names for easy identification
3. Test with small examples before large instances
4. Review generated scripts for reasonableness

### For Problem Authors
1. Create comprehensive `json-schema.md` documentation
2. Provide diverse example instances
3. Include `example.dzn` and `example.json` for reference
4. Ensure MiniZinc model is well-documented

## Integration

The agent integrates with the broader LMTune ecosystem:

- **Input**: Uses `dzn-to-json` for data conversion
- **Output**: Compatible with `lmtune-run` for script execution
- **Storage**: Archives in problem directories for organization
- **Analysis**: Generated scripts can be studied and modified manually

## Future Enhancements

Planned improvements include:
- Multi-instance batch analysis
- Parameter sensitivity analysis
- Historical performance tracking
- Cross-problem pattern recognition
- Automated hyperparameter optimization