Exploring Behavior-Driven Development for Code Generation

ACL ARR 2025 February Submission443 Authors

Abstract:

Code generation has long been a challenging task in natural language processing, with existing models often struggling to produce correct and functional code solutions.
This paper explores integrating Behavior-Driven Development (BDD)—a user-centric agile methodology—into the code generation process.
We propose BDDCoder, a novel multi-agent framework comprising four roles: Programmer, Tester, Requirements Analyst, and User, designed to simulate real-world BDD workflows.
BDDCoder consists of two variants: BDD-NL, which uses natural language scenarios for code generation and LLM-based self-validation
and BDD-Test, which converts scenarios into executable test cases for code validation.
Through empirical evaluation on benchmark datasets (HumanEval, MBPP, and their EvalPlus variants), we demonstrate that
BDD-NL with LLM self-validation would hinder code generation performance,
while BDD-Test significantly outperforms BDD-NL, achieving up to a 15.1\% improvement in pass@1 scores.
Our findings highlight the potential of BDD to enhance requirement clarity and code alignment with user needs,
offering a robust framework for future research on integrating software engineering methodologies into automated code generation.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: code generation and understanding

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 443