SMAR + NIE IdeaGen: A knowledge graph based node importance estimation with analogical reasoning on large language model for idea generation

Olaide Nathaniel Oyelade, Hui Wang, Karen Rafferty

Abstract: Idea generation describes a creative process involving reasoning over some knowledge to derive new information. Traditional approaches such as mind-map and brainstorming are limited and often fail due to lack of quality ideas and ineffective methods. The reasoning capability of large language models (LLMs) have been investigated for ideation tasks and have reported interesting performance. However, these models suffer from limited logical reasoning capability which hinders the use of structural and factual real-world knowledge in discovery of latent insight and predict possible outcome when applied to ideation. In addition, the possibility of LLMs regurgitating knowledge learnt from datasets might adversely impact the degree of novel ideas the models can generate. In this paper, a two-stage logical reasoning approach is applied to initiate the search for candidate idea pathways based on the knowledge graphs (KGs) to address the problem of reasoning, domain-specificity and novelty. The divergence stage this reasoning explores utilizes a new node importance estimation (NIE) technique over KGs to discover latent connections supporting idea generation. In the convergence stage of this reasoning, subgraph matching using analogical reasoning (SMAR) is applied to find matching patterns to describe a new idea. The use of SMAR + NIE and KGs helps to achieve an improvement in reasoning over KGs before transferring such reasoning to LLMs for translation of idea into natural language. To evaluate the degree of novelty of ideas generated, a relevance-to-novelty scoring metrics is proposed based on multiple premise entailment (MPE). We combined this metric with other popular metrics to evaluate the performance of SMAR + NIE on benchmark datasets, and as well on the quality of ideas generated. Findings from the study showed that this approach demonstrates competitive performance with mainstream LLMs in idea generation tasks.