# 🌐 WebAggregator: Scaling Complex Logical Information Aggregation for Web Agent Foundation Models

## Introduction

- **WebAggregator** aims to generate diverse, high-quality training data for web agent foundation models, enhancing their capabilities in multi-tool usage, **information seeking**, and **information aggregation**.

- The project demonstrates strong performance on GAIA-text and the WebAggregator test set.


## ✨ Features

- 🤖 **Fully Automated and Verifiable QA Construction**  
- 😄 **Open Source**: Complete codebase including QA construction engine, queries, trajectories, and models.
- 👍 **Highly Customizable**: Collect data tailored to your needs with minimal human effort, and easily customize your own agent!


## ⚡ Quick Start

Follow these steps to get started:

### 1️⃣ Clone the Repository

```bash
git clone <repository_url>
```

### 2️⃣ Install smolagents Dependencies

1. This project builds upon smolagents’ “open deep research” example. Please follow their documentation to install required dependencies:  
   👉 [smolagents open_deep_research dependencies](https://github.com/huggingface/smolagents/tree/main/examples/open_deep_research)

2. Then, install this project’s additional requirements:

```bash
pip install -r requirements.txt
```

---

## 🚀 Usage

### ⚙️ Model Configuration

Edit `./config.py` to Configure your agent’s foundation LLM and LLMs supporting specific tools.

---

### ▶️ Running the Project

> **Note:** Before running any scripts, ensure all paths, model checkpoints, and other necessary parameters are properly set in the source files.

---

#### 1️⃣ Evaluation

To run evaluation:

```bash
bash run/test.sh
```

This command evaluates your specified model and benchmark. After evaluation, it uses an LLM as judge to assess performance and prints the accuracy.

---

#### 2️⃣ QA Construction

Start building automatic web agent data:

1. Download our collected URLs 👉 [URLs](https://huggingface.co/datasets/Ray121381/WebAggregatorQA) or gather URLs related to your domains of interest.

2. Then, run the following command to collect the data.

```bash
bash run/QA_building.sh
```

---

#### 3️⃣ Trajectory Sampling

Collect trajectories to fine-tune your own agent foundation models:

```bash
bash run/traj_sampling.sh
```

---

#### 4️⃣ Evaluate Your Foundation Models

Serve your checkpoint through vLLM and update the corresponding settings in `config.py`. Make sure the correct `model_id` is set in the evaluation script, then run:

```bash
bash run/test.sh
```


---

## Citation

```bibtex
@misc{webagg,
  author = {},
  title = {WebAggregator},
  year = {2025},
  howpublished = {}
}
```
