# Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Tasks

[![Python](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Poetry](https://img.shields.io/badge/dependency_management-poetry-blue.svg)](https://python-poetry.org/)

**Orchestrator** is a multi-agent coordination framework that leverages attention-inspired self-emergent coordination and reflective benchmarking to optimize global task performance in complex, non-linear environments. This implementation demonstrates the framework's capabilities through challenging maze-solving tasks with varying difficulty levels.

## 🧩 Research Overview

The Orchestrator framework addresses key limitations in multi-agent systems (MAS) operating in long-horizon, partially observable environments. By integrating **active inference principles** with **dynamic orchestration**, the system enables agents to:

- **Minimize variational free energy** through real-time performance benchmarking
- **Overcome partial observability** via coordinated agent-to-agent and agent-to-environment interactions
- **Adapt dynamically** to rising complexity levels in extended task sequences
- **Scale efficiently** using lightweight LLM models suitable for production deployment

### Key Contributions

1. **Self-Optimizing Cell Architecture**: Modular design with planning, execution, and observation instances driven by active inference feedback
2. **Reflective Benchmarking Mechanism**: Real-time coordination optimization using variational free energy metrics
3. **Production-Ready Performance**: Sustained task-completion accuracy (75-100%) across various difficulty levels using compact LLMs

## 🏗️ System Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    Orchestrator Framework                   │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐    │
│  │   Planning  │    │ Execution   │    │Orchestration│    │
│  │    Node     │───▶│   Nodes     │◀──▶│    Node     │    │
│  │             │    │   (n=2)     │    │             │    │
│  └─────────────┘    └─────────────┘    └─────────────┘    │
│                             │                             │
│                             ▼                             │
│               ┌─────────────────────────┐                 │
│               │ Active Inference Engine │                 │
│               │ • Epistemic Uncertainty │                 │
│               │ • Pragmatic Costs       │                 │
│               │ • Dynamic Weight Updates│                 │
│               └─────────────────────────┘                 │
└─────────────────────────────────────────────────────────────┘
```

## 🚀 Quick Start

### Prerequisites

- Python 3.12+
- [Poetry](https://python-poetry.org/docs/#installing-with-the-official-installer) for dependency management
- OpenAI API key

### Installation

1. **Clone the repository**
   ```bash
   git clone https://github.com/your-org/multi-agent-maze-runner.git
   cd multi-agent-maze-runner
   ```

2. **Set up Python environment**
   ```bash
   # Create virtual environment
   python -m venv .venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate
   ```

3. **Install dependencies**
   ```bash
   # Install Poetry if not already installed
   curl -sSL https://install.python-poetry.org | python3 -
   
   # Install project dependencies
   poetry install
   ```

4. **Configure environment**
   ```bash
   # Copy and configure environment variables
   cp .env.example .env
   # Edit .env and add your OPENAI_API_KEY
   ```

5. **Run a test maze**
   ```bash
   poetry run main
   ```

## 🎮 Usage

### Basic Execution

Run the system on a specific maze:

```bash
cd orchestrator_maze_implementation
poetry run python main.py
```

### Configuration

Modify `orchestrator_maze_implementation/config.yaml` to customize:

```yaml
# Maze selection (20 pre-built mazes available)
maze:
  uuid: "d1d68192f1394d4d"  # Change to different maze ID

# Agent configuration
agents:
  global:
    num_execution_agents: 2  # Number of execution agents
    steps_per_agent: 5       # Steps per agent before rotation
  
  execution:
    model: "gpt-4.1-nano"    # LLM model for execution agents
    temperature: 0.1

# Ablation controls
ablations:
  enable_orchestration_agent: true
  enable_dynamic_weights: true
  enable_teammate_coordination: true
```

### Batch Experiments

For large-scale experiments and parameter sweeps:

```bash
# Generate configurations for all mazes and temperature variations
python generate_configs.py

# Run batch experiments with parallelization
python batch_runner.py --parallel 4 --timeout 900

# Debug specific configuration
python batch_runner.py --debug --debug_config config_maze_[ID]_temp_[TEMP].yaml
```

See [`README_BATCH_RUNNER.md`](orchestrator_maze_implementation/README_BATCH_RUNNER.md) for detailed batch processing documentation.

## 📊 Experimental Results

Our evaluation demonstrates significant performance improvements:

| Configuration | Easy Mazes | Medium Mazes | Hard Mazes |
|---------------|------------|--------------|------------|
| Solo Agent   | 32.35%      | 30.30%        | 0.0%       |
| + FE Benchmarking | 100.0% | 72.2%        | 84.62%      |
| + Orchestration | 100.0%   | 100.0%       | 71.88%      |

**Key Findings:**
- **3.03x improvement** factor compared to baseline solo agents
- **100% success rate** on medium difficulty mazes with full framework
- **Resource-efficient**: Achieves high performance with compact LLMs (GPT-4.1-nano & GPT-5-nano)

## 🔬 Technical Details

### Active Inference Implementation

The system implements variational free energy optimization:

```
F(n,t,k) = U_epistemic(n,t,k) - C_accuracy(n,t,k)
```

Where:
- **U_epistemic**: Information entropy between consecutive states
- **C_accuracy**: Behavioral efficiency and coordination costs
- **Dynamic weights**: Real-time policy adjustments based on performance classification

### Maze Complexity

Evaluation uses Shannon entropy-based complexity metrics:
- **Surprisingness**: Entropy of directional decisions along optimal path
- **Deceptiveness**: Entropy of trap transitions leading to suboptimal paths
- **Trap Complexity**: Weighted dead-end path analysis

## 📁 Project Structure

```
multi-agent-maze-runner/
├── orchestrator_maze_implementation/    # Core framework implementation
│   ├── agents/                         # Agent implementations
│   │   ├── maze_execution_agent.py     # Execution node logic
│   │   ├── maze_orchestration_agent.py # Orchestration node logic
│   │   └── benchmarking_node.py        # Active inference benchmarking
│   ├── config/                         # Configuration management
│   ├── mazes/                          # Pre-built maze collection (20 mazes)
│   ├── router/                         # Workflow routing logic
│   ├── state/                          # System state management
│   ├── tools/                          # Agent tool interfaces
│   ├── visualization/                  # Maze visualization components
│   ├── main.py                         # Main execution entry point
│   ├── batch_runner.py                 # Batch processing system
│   └── config.yaml                     # Default configuration
├── results/                            # Experimental results
└── README_BATCH_RUNNER.md              # Batch processing documentation
```

## 🧪 Research Applications

This framework is designed for research in:

- **Multi-agent coordination** in complex environments
- **Active inference** applications in AI systems
- **Long-horizon planning** and task execution
- **Partially observable environments** and uncertainty handling
- **Resource-efficient AI** for production deployments

## 📖 Citation

If you use this work in your research, please cite:

```bibtex
@article{<neurips-anonymous-review>2025orchestrator,
  title={Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Tasks},
  journal={NeurIPS Workshop on Multi-Turn Interactions in Large Language Models},
  year={2025}
}
```

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 🔗 Links

- [Documentation](orchestrator_maze_implementation/README_BATCH_RUNNER.md)

---

**Built with ❤️ for advancing multi-agent coordination research**