# 🏟️ Long Code Arena Baselines

## What is Long Code Arena? 

[**Long Code Arena**](https://huggingface.co/spaces/JetBrains-Research/long-code-arena) is a suite of benchmarks for code-related tasks with large contexts, up to a whole code repository.
It currently spans six different tasks and contains six datasets:
* 🤗 [Library-based code generation](https://huggingface.co/datasets/JetBrains-Research/lca-library-based-code-generation)
* 🤗 [CI builds repair](https://huggingface.co/datasets/JetBrains-Research/lca-ci-builds-repair)
* 🤗 [Project-level code completion](https://huggingface.co/datasets/JetBrains-Research/lca-project-level-code-completion)
* 🤗 [Commit message generation](https://huggingface.co/datasets/JetBrains-Research/lca-commit-message-generation)
* 🤗 [Bug localization](https://huggingface.co/datasets/JetBrains-Research/lca-bug-localization)
* 🤗 [Module summarization](https://huggingface.co/datasets/JetBrains-Research/lca-module-summarization)

## Where are the baselines? 

For each task, we have different approaches and environments. You can find the baseline solutions and the necessary information about each task in the corresponding directories. 

## How can I submit my results? 

We are excited to invite you to participate in solving our [benchmarks]((https://huggingface.co/spaces/JetBrains-Research/long-code-arena))! To submit your results, please send the following materials to our 📩 email (lca@jetbrains.com):  
* **Results**: Include the summary of your benchmark outcomes.
* **Reproduction Package**: To ensure the integrity and reproducibility of your results, please include the code for context collection (if any), generation of predictions, and evaluating. You can follow [our baselines](https://github.com/JetBrains-Research/lca-baselines) as a reference.  
* **Metadata**: Model information, organization name, licence of your model, context size, and other information you find relevant.
