LLM-based Dynamic Differential Testing for Database Connectors with Reinforcement Learning-Guided Prompt Selection

Ce Lyu, Yanhao Wang, Jie Liang, Minghao Zhao

Abstract: Database connectors are critical components that enable applications to interact with database management systems (DBMS) but their security vulnerabilities are often neglected. Unlike traditional software defects, connector vulnerabilities exhibit subtle behavioral patterns and are inherently challenging to detect. Moreover, non-standardized implementation of connectors leaves potential risks (i.e., unsafe implementations) but is more elusive. As a result, existing fuzzing methods are ineffective in finding such vulnerabilities. Even large language model (LLM)-based methods are still incapable of generating test cases that can invoke all the interface and internal logic of database connectors due to a lack of domain knowledge.In this paper, we propose a new LLM-based test case generation method guided by reinforcement learning (RL) for database connector testing. Specifically, to equip the LLM with sufficient and appropriate domain knowledge, a parameterized template is composed for prompt construction. The LLM then generates test cases instructed by the constructed prompts, which are dynamically evaluated through differential testing across multiple connectors. The testing process is carried out iteratively, where RL is adopted to select the optimal prompt in each round based on behavioral feedback from the previous rounds, to maximize the efficiency of discovering inconsistencies. Finally, we implement and evaluate the aforementioned methodology on two widely used JDBC connectors, namely MySQL Connector/J and OceanBase Connector/J. In the preliminary results, we have reported 16 bugs, among which 10 are officially confirmed, and the rest are acknowledged as unsafe implementations.

External IDs:dblp:conf/kbse/LyuWLZ25