Reasoning about concepts with LLMs: Inconsistencies abound

Rosario Uceda Sosa, Karthikeyan Natesan Ramamurthy, Maria Chang, Moninder Singh

Research Area: Data, Evaluation, Inference algorithms for LMs, LMs for everyone, LMs and interactions

Keywords: KG reasoning in LLMs, LLM consistency, Synthetic data generation for LLM evaluation, RAG, prompt engineering,

TL;DR: Through systematic testing of simple knowledge graphs, we've discovered plenty of inconsistent reasoning in LLMs. Here we discuss the test results and ways to make the LLMs more robust.

Abstract:

The ability to summarize and organize knowledge into abstract concepts is key to learning and reasoning. Many industrial applications rely on the consistent and systematic use of concepts, especially when dealing with decision-critical knowledge. However, we demonstrate that, when methodically questioned, large language models (LLMs) often display and demonstrate significant inconsistencies in their knowledge.

Computationally, the basic aspects of the conceptualization of a given domain can be represented as Is-A hierarchies in a knowledge graph (KG) or ontology, together with a few properties or axioms that enable straightforward reasoning. We show that even simple ontologies can be used to reveal conceptual inconsistencies across several LLMs. We also propose strategies that domain experts can use to evaluate and improve the coverage of key domain concepts in LLMs of various sizes. In particular, we have been able to significantly enhance the performance of LLMs of various sizes with openly available weights using simple knowledge-graph (KG) based prompting strategies.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the COLM Code of Ethics on https://colmweb.org/CoE.html

Author Guide: I certify that this submission complies with the submission instructions as described on https://colmweb.org/AuthorGuide.html

Submission Number: 262