Consistency of Code: A Prompt Based Approach to Comprehend Functionality

Hoyoung Choi, Hyunjae Park, Young-June Choi, Kyungsik Han

Abstract: Large language model (LLM)-based AI for code model (e.g., Copilot) demonstrates the potential of using AI in specialized domains such as software engineering. While previous research has focused on fine-tuning models with additional data and computational cost to construct models optimized for specific domains, our research focuses on prompt engineering methods that maximize the performance of existing models. We conducted a quantitative and qualitative user study using the AI for code model and identified two limitations that hinder the recommendation performance of the model. We propose two methods to address these limitations through effective prompt engineering. Finally, we identified the potential for the use of our proposed methods to be utilized and discussed the direction of future research for the effective use of the LLM.