# 🥷 BenchName Baselines

## What is BenchName? 

[**BenchName**](https://huggingface.co/spaces/anon-iclr-submission/benchname) is a suite of benchmarks for code-related tasks with large contexts, up to a whole code repository.
It currently spans six different tasks and contains six datasets:
* 🤗 [Library-based code generation](https://huggingface.co/datasets/anon-iclr-submission/benchname-library-based-code-generation)
* 🤗 [CI builds repair](https://huggingface.co/datasets/anon-iclr-submission/benchname-ci-builds-repair)
* 🤗 [Project-level code completion](https://huggingface.co/datasets/anon-iclr-submission/benchname-project-level-code-completion)
* 🤗 [Commit message generation](https://huggingface.co/datasets/anon-iclr-submission/benchname-commit-message-generation)
* 🤗 [Bug localization](https://huggingface.co/datasets/anon-iclr-submission/benchname-bug-localization)
* 🤗 [Module summarization](https://huggingface.co/datasets/anon-iclr-submission/benchname-module-summarization)

## Where are the baselines? 

For each task, we have different approaches and environments. You can find the baseline solutions and the necessary information about each task in the corresponding directories. 

## How can I submit my results? 

We are excited to invite you to participate in solving our [benchmarks]((https://huggingface.co/spaces/anon-iclr-submission/benchname))! To submit your results, please send the following materials to our 📩 email (benchname@anon.com):  
* **Results**: Include the summary of your benchmark outcomes.
* **Reproduction Package**: To ensure the integrity and reproducibility of your results, please include the code for context collection (if any), generation of predictions, and evaluating. You can follow [our baselines](https://github.com/anon-iclr-submission/benchname-baselines) as a reference.  
* **Metadata**: Model information, organization name, licence of your model, context size, and other information you find relevant.
