CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationDownload PDF

Published: 29 Jul 2021, Last Modified: 24 May 2023NeurIPS 2021 Datasets and Benchmarks Track (Round 1)Readers: Everyone
Abstract: Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. CodeXGLUE also features three baseline systems, including the BERT-style, GPT-style, and Encoder-Decoder models, to make it easy for researchers to use the platform. The availability of such data and baselines can help the development and validation of new methods that can be applied to various program understanding and generation problems.
Supplementary Material: zip
Contribution Process Agreement: Yes
Dataset Url:
License: Our codes follow MIT License. Our datasets follow Computational Use of Data Agreement (C-UDA) License.
Author Statement: Yes
13 Replies