Chain-of-Trigger Backdoor (CoTri)

Overview
--------
This repository CoTri on the WebShop environment. 
It provides:
- Text and vision modality evaluation scripts and original data
- An analyzer to (1) define malicious objectives, (2) interact with WebShop to discover expert-like trajectories, and (3) extract unique trigger tokens from observations
- A noise-configurable WebShop runtime environment
- A utility to convert HTML observations to images for vision experiments

Key Files and Directories
-------------------------
- `webshop_train.json`, `webshop_test.json`: original expert training/test trajectories
- `webshop_analyzer.py`: used to
  1) define a malicious objective (e.g., “buy the most expensive item”),
  2) interact with the environment to discover expert trajectories, and
  3) extract unique tokens from those trajectories as triggers
- `text_cotri_eval.ipynb`: evaluates CoTri backdoor on the text modality
- `qwenvl_cotri_eval.py`: evaluates CoTri on the vision modality 
- `simple_html_to_image.py`: converts WebShop HTML observations into images for vision models
- `Noisy_Webshop/`: a noise-augmented WebShop environment; all noise switches are controlled in
  `Noisy_Webshop/agentenv-webshop/agentenv_webshop/environment.py`
- `Noisy_Webshop/agentenv/eval_webshop_performance.py`: evaluates model performance under different WebShop noise settings



Path Note: `webshop_analyzer.py` adds `Noisy_Webshop/agentenv-webshop/webshop` to `sys.path`, so you can run it without installing as a package. To install as a package, follow the `agentenv-webshop` README.

Quick Start
-----------
1) Run the noise-enabled WebShop environment (local simulation)
- Noise switches and parameters: `Noisy_Webshop/agentenv-webshop/agentenv_webshop/environment.py`
- Configure random feedback, null feedback, etc.

2) Discover triggers with the analyzer
```bash
python webshop_analyzer.py
```
This will:
- Search and select products (e.g., the most expensive)
- Log interaction trajectories
- Extract unique trigger token combinations from observations and save to `webshop_analysis_*.json`

3) Evaluate CoTri on text modality
- Open `text_cotri_eval.ipynb` and run cells to compute backdoor metrics (e.g., ASR, FTR, CR)

4) Evaluate CoTri on vision modality (e.g., QwenVL)
```bash
python qwenvl_cotri_eval.py
```
To produce vision inputs from HTML:
```bash
python simple_html_to_image.py --html path/to/input.html --out path/to/output.png
```

Noise Controls (Noisy_Webshop)
------------------------------
All noise is configured in `Noisy_Webshop/agentenv-webshop/agentenv_webshop/environment.py`:
- Random feedback: `enable_random_feedback`, `random_feedback_probability`
- Randomization modes: `randomization_mode ∈ {replace, append, prefix, typo, null}`
- One-time randomization: `one_time_random`
- Null feedback: `enable_null_feedback`, `null_feedback_rounds`, `null_feedback_probability`
- Random keyword pool: `random_search_keywords`

To expose the environment via an HTTP server:
```bash
cd Noisy_Webshop/agentenv-webshop
PYTHONPATH=Noisy_Webshop/agentenv-webshop/webshop \
python -c "import uvicorn; from agentenv_webshop.server import app; uvicorn.run(app, host='127.0.0.1', port=3000, log_level='info')"
```


Data
----
- `webshop_train.json`: expert training trajectories
- `webshop_test.json`: expert test trajectories




