# CoT Obfuscation Experiments

This repository contains code to reproduce Chain-of-Thought (CoT) obfuscation and instruction-following experiments.

## Contents
- `cot_obfuscation/instruction_following.py`: instruction-following microtasks with internal/external CoT handling, provider adapters, and batched monitors.
- `cot_obfuscation/obf_few_shots.py`: few-shot example builders used as obfuscation cues.
- `cot_obfuscation/obfuscation.py`: obfuscation experiments on toy tasks; builds pressure prompts, runs models, and evaluates with two monitors.

## Requirements
Python 3.10+

Install dependencies:
```bash
pip install -r requirements.txt
```

## Environment Variables
Set provider keys as needed:
- `OPENAI_API_KEY` (required for monitors and OpenAI models)
- `ANTHROPIC_API_KEY` (for Anthropic models)
- `DEEPINFRA_API_KEY` (for DeepInfra-hosted models: DeepSeek, Qwen, GPT-OSS)
- `MISTRAL_API_KEY` (for Mistral/Magistral models)

## Instruction-Following (Microtasks)
Run a single instruction across tasks and models by editing the main block or importing the API:
```python
from cot_obfuscation.instruction_following import steering_catalog, RunConfig, run_single_instruction_experiments

instr = steering_catalog()["language"][0]  # e.g., lang_pl
cfg = RunConfig(
    model_slug="anthropic/claude-3-7-sonnet-20250219",
    runs_per_instruction=8,
    internal_reasoning=True,
    temperature=1.0,
    reasoning_effort="medium",
    reasoning_batch_size=8,
    monitor_batch_size=16,
    monitor_model="gpt-5-mini",
)
results = run_single_instruction_experiments(instr, cfg)
```
Outputs are plain Python objects; you can serialize with the provided helpers.

## Obfuscation Experiments (Toy Tasks)
Basic usage from Python:
```python
from cot_obfuscation.obfuscation import run_obfuscation_experiments
from cot_obfuscation.instruction_following import RunConfig

cfg = RunConfig(
  model_slug="anthropic/claude-3-7-sonnet-20250219",
  runs_per_instruction=3,
  internal_reasoning=True,
  temperature=1.0,
  reasoning_effort="medium",
  reasoning_batch_size=8,
  monitor_batch_size=16,
  monitor_model="gpt-5-mini",
)
results = run_obfuscation_experiments(cfg, include_sensitive=False)
```
Save/load JSON:
```python
from cot_obfuscation.obfuscation import save_obfuscation_results_to_json, read_obfuscation_results_from_json
save_obfuscation_results_to_json(results, "results/obfuscation_run.json")
loaded = read_obfuscation_results_from_json("results/obfuscation_run.json")
```

### Settings (Prompt Pressure)
See `obfuscation.py` `all_settings` for available pressure stacks. You can pass `setting_names=[...]` to select a subset.

## SHADE‑Arena Experiments
We also ran experiments in the SHADE‑Arena environments. We do not include code for these experiments because the environments are not public; access was provided to us upon request. The exact prompts we used for SHADE‑Arena are available in the paper appendix.

## Notes
- Final answers must appear inside `<answer>...</answer>`.
- For models without native hidden traces, external `<think>...</think>` tags are used to extract internal reasoning.
- Monitoring prompts expect JSON responses; automatic retry is implemented for parse failures. 