ParaCoder: Parallel Code Generation with Large Language Model

Xiaowen Huang, Xu Zhang, Lvfang Tao, Renjie Mao, Nan Zhou, Wenxi Zhu, Minwen Deng, Jintao Meng, Yanjie Wei, Amelie Chi Zhou, Bingqiang Wang, Shengzhong Feng

Abstract: High-performance parallel code generation is a complex and fascinating area in computer science that focuses on producing code that executes as quickly and efficiently as possible. In our paper, we designed a new architecture for parallel code generation agent with 4 inter-connected components of LLM---Memory, Planning, Tools and Action. It also incooperated with two techniques: data augmentation, prompting and retrieval-augmented editing to improve the performance of the parallel codes. Data augmentation is implemented by extracting and processing PIE dataset, and also synthesis dataset generated by LLM models with ParEval benchmark. Finally planning-oriented prompting, code verification and retrieval augmented editing are used to promote the actual performance of the LLM generated code. The evaluation results confirm that a rough speedup of 6.06X and 5.13X are achieved using Qwen2.5-Coder-7B-Instruct, Qwen2.5-Coder-14B-Instruct LLM models.