# Agent Misalignment Evaluations: Household Management

## Usage

First, install the `inspect_ai` and `inspect_evals` Python packages with:

```bash
pip install inspect_ai
pip install git+https://github.com/UKGovernmentBEIS/inspect_evals
```

Then, evaluate against one or more models with:

```bash
inspect eval inspect_evals/household_management --model openai/gpt-4o-mini
```

More examples:
```bash
# Run with all models and temperature 0
inspect eval inspect_evals/household_management \
    --model openai/gpt-4.1-2025-04-14,anthropic/claude-3-7-sonnet-20250219,google/gemini-2.5-pro-preview-05-06,openai/o4-mini-2025-04-16,anthropic/claude-3-opus-20240229,google/gemini-2.0-flash-001 \
    --temperature 0

# Test launch 1 sample, 1 model
inspect eval inspect_evals/household_management \
    --model openai/gpt-4o-mini \
    --temperature 0 \
    --sample-id 1-3_rapid_executor

# Test launch 1 sample, 1 model, and the CMSScorer
inspect eval inspect_evals/household_management \
    --model openai/gpt-4o-mini \
    --temperature 0 \
    --sample-id 1-3_rapid_executor \
    -T cms_scorer=True

# Test launch a run specific scorer from an already existing .eval file
inspect score src/inspect_evals/agentmisalignment/evals/household_management/results/logs/household_management_claude_3_7_sonnet.eval \
    --scorer src/inspect_evals/agentmisalignment/scorers/household_management_scorer.py -S answer_idx=0
```