Unpacking Large Language Models with Conceptual Consistency

Pritish Sahu, Michael Cogswell, Yunye Gong, Ajay Divakaran

Keywords: Conceptual Consistency, Theory of Mind, Zero Shot Prompting, Large Language Models, Semantic Consistency, Unsupervised Question Answering, Background Knowledge Extraction

Abstract:

If a Large Language Model (LLM) answers “yes” to the question “Are moun-
tains tall?” then does it know what a mountain is? Can you rely on it respond-
ing correctly or incorrectly to other questions about mountains? The success of
Large Language Models (LLMs) indicates they are increasingly able to answer
queries like these accurately, but that ability does not necessarily imply a general
understanding of concepts relevant to the anchor query. We propose conceptual
consistency to measure a LLM’s understanding of relevant concepts. This novel
metric measures how well a model can be characterized by finding out how con-
sistent its responses to queries about conceptually relevant background knowledge
are. To compute it we extract background knowledge by traversing paths between
concepts in a knowledge base and then try to predict the model’s response to the
anchor query from the background knowledge. We investigate the performance
of current LLMs in a commonsense reasoning setting using the CSQA dataset
and the ConceptNet knowledge base. While conceptual consistency, like other
metrics, does increase with the scale of the LLM used, we find that popular mod-
els do not necessarily have high conceptual consistency. Our analysis also shows
significant variation in conceptual consistency across different kinds of relations,
concepts, and prompts. This serves as a step toward building models that humans
can apply a theory of mind to, and thus interact with intuitively.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)

TL;DR: Conceptual consistency measures whether knowledge of relevant background information is consistent with ability to answer questions correctly in large language models.