Effi-Code: Unleashing Code Efficiency in Language Models

Dong HUANG, Guangtao Zeng, Jianbo Dai, Meng Luo, Han Weng, Yuhao QING, Heming Cui, Zhijiang Guo, Jie Zhang

Keywords: Large Langugae Models, Code Generation, Program Synthesis, Efficient Method, Alignment

TL;DR: Effi-Code generates high-quality, efficient code samples, which are then used for instruction tuning to significantly improve both code correctness and efficiency across various LLMs.

Abstract:

As the use of large language models (LLMs) for code generation becomes more prevalent in software development, it is critical to enhance both the efficiency and correctness of the generated code.
Existing methods and models primarily focus on the correctness of LLM-generated code, ignoring the efficiency.
In this work, we present Effi-Code, an approach to enhancing code generation in LLMs that can improve both efficiency and correctness. We introduce a Self-Optimization process based on Overhead Profiling that leverages open-source LLMs to generate a high-quality dataset of correct and efficient code samples. This dataset is then used to fine-tune various LLMs. Our method involves iterative refinement of generated code, guided by runtime performance metrics and correctness checks. Extensive experiments demonstrate that models fine-tuned on the Effi-Code significantly improve code correctness and efficiency across task types.
For example, the pass@1 of DeepSeek-Coder-6.7B-Instruct generated cod increases from **43.3\%** to **76.8\%**, and the average execution time for the same correct tasks decreases by **30.5\%**.
Effi-Code offers a scalable and generalizable approach to improving code generation in AI systems, with potential applications in software development, algorithm design, and computational problem-solving.

Primary Area: foundation or frontier models, including LLMs

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2292