# OmniParser: Screen Parsing tool for Pure Vision Based GUI Agent


**OmniParser** is a comprehensive method for parsing user interface screenshots into structured and easy-to-understand elements, which significantly enhances the ability of GPT-4V to generate actions that can be accurately grounded in the corresponding regions of the interface. 

## Install 
```python
conda create -n "omni" python==3.12
pip install -r requirements.txt
```

## Examples:
We put together a few simple examples in the demo.ipynb. 

## Gradio Demo
To run gradio demo, simply run:
```python
python gradion_demo.py
```

## the eval code for the benchmarks are included in corresponding python script
```python
python omniparser_eval_aitw_simplified_parallel.py
python omniparser_eval_m2w_aitw_simplified.py
python omniparser_eval_parsed_mind2web.py
python omniparser_eval_seeassign.py
python omniparser_eval_seeclick.py
```