Optimizing KBQA by Correcting LLM-Generated Non-Executable Logical Form Through Knowledge-Assisted Path Reconstruction

Ranran Bu, Jianqi Gao, Jian Cao, Hongming Cai, Jinghua Tang, Yonggang Zhang

Abstract: Knowledge base question answering (KBQA) refers to the task of answering natural language questions using factual information from large-scale knowledge bases (KBs). To obtain accurate answers, recent research optimizes semantic parsing methods, a major KBQA approach, with large language models (LLMs), where concise logical forms (LFs) are generated by LLMs and executed in KBs. Although these methods demonstrate superior performance, they still encounter the problem that some generated LFs fail to yield answers when executed, significantly limiting their effectiveness. To mitigate this issue, we propose KARV, a Knowledge-Assisted reasoning path Reconstruction and hierarchical Voting approach for non-executable LFs. This method extracts semantic knowledge from KBs as guidance to correct and reconstruct reasoning paths, deriving answers through a voting-based strategy. The insight is that non-executable LFs generated by LLMs still contain rich semantic information, and the knowledge retrieved from KBs can effectively correct them. Specifically, we fine-tune LLMs to generate high-quality LFs, and the nonexecutable LFs are decomposed into multiple path branches based on mentioned entities. Semantic knowledge from KBs is then leveraged to correct the entities and relations within these branches, effectively reconstructing the reasoning paths. To obtain precise final answers, we apply a hierarchical voting strategy both within and across the non-executable LFs. Our proposed method achieves state-of-the-art performance on benchmarks including WebQuestionSP (WebQSP), ComplexWebQuestions (CWQ), and FreebaseQA.

External IDs:dblp:journals/tkde/BuGCCTZ26