ProtocolLLM: RTL Benchmark for SystemVerilog Code Generation of Communication Protocols

Arnav Miteshkumar Sheth, Ivaxi Sheth, Mario Fritz

Presentation: In-Person

Keywords: Large Language Models, HDL Generation, SystemVerilog, Communication Protocols

Presenter Full Name: Arnav Miteshkumar Sheth

TL;DR: We introduce a benchmark to evaluate LLMs on communication protocol generation in HDL, revealing limitations in synthesis and functional correctness

Presenter Email: [e-mail]

Abstract: Recent advances in Large Language Models have shown promising capabilities in generating code for general-purpose programming languages. In contrast, their applicability for hardware description languages, particularly for generating synthesizable and functionally correct designs, remains significantly underexplored. HDLs such as SystemVerilog are logic-oriented and demand strict adherence to timing semantics, concurrency, and synthesizability constraints. Moreover, HDL-based design flows encompass a broad set of tasks beyond structural code generation, including testbench development, assertion-based verification, timing closure, and protocol-level integration for on-chip communication. The objective of our paper is to analyze the capabilities of state-of-the-art LLMs in generating SystemVerilog implementations of standard communication protocols, a core component of embedded and SoC architectures. This paper introduces the first benchmark suite targeting four widely used protocols—SPI, I²C, UART, and AXI. We define multiple code generation tasks that capture varying levels of design abstraction and prompt specificity. The generated designs are assessed for syntactic correctness, synthesizability, and functional fidelity via waveform simulation and test benches.

Presenter Bio: Arnav Sheth a Computer Engineering senior at UIUC interested in building experience in efficient hardware design, microarchitecture, FPGA-based sensor integration and AI-based HW design..

Paper Checklist Guidelines: I certify that all co-authors have validated the presented results and conclusions, and have read and commit to adhering to the Paper Checklist Guidelines, Call for Papers and Publication Ethics.

YouTube Link: N/A

YouTube Link Poster: N/A

Dataset Release: I certify that all co-authors commit to release the dataset and necessary scripts to reproduce the presented results.

Google Slides: https://docs.google.com/presentation/d/1ZUUvw5mAPBRld2psU3AP_PWd2ZKAzix61Ov5GBLtHBU/edit?usp=sharing

Poster: Yes

Workshop Registration: Yes, the presenter has registered for the workshop.

YouTube Link Short: N/A

Submission Number: 18