HoneyComb: A Flexible LLM-Based Agent System for Materials Science

Huan Zhang, Yu Song, Ziyu Hou, Santiago Miret, Bang Liu

Submission Track: LLMs for Materials Science - Full Paper

Submission Category: All of the above

Keywords: large language model (LLM), tool-based LLM, retriever augmented language models, materials science, LLM agent

TL;DR: We propose HoneyComb, an agent-based framework for LLMs for materials science leveraging knowledges bases and tools, and show that HoneyComb improves the performances of various LLMs and materials science tasks.

Abstract: The emergence of specialized large language models (LLMs) has shown promise in addressing complex tasks in materials science. Many LLMs, however, often struggle with the distinct complexities of materials science tasks, such as computational challenges, and rely heavily on outdated implicit knowledge, leading to inaccuracies and hallucinations. To address these challenges, we introduce HoneyComb, the first LLM-based agent system specifically designed for materials science. HoneyComb leverages a reliable, high-quality materials science knowledge base (MatSciKB) and a sophisticated tool hub (ToolHub) tailored specifically for materials science to enhance its reasoning and computational capabilities. MatSciKB is a curated, structured knowledge collection based on reliable literature, while ToolHub employs an Inductive Tool Construction method to generate, decompose, and refine API tools for materials science. Additionally, HoneyComb leverages a retriever module that adaptively selects the appropriate knowledge source or tools for specific tasks, thereby ensuring accuracy and relevance. Our results demonstrate that HoneyComb significantly outperforms baseline models across various tasks in materials science, effectively bridging the gap between current LLM capabilities and the specialized needs of this domain. Furthermore, our adaptable framework can be easily extended to other scientific domains, highlighting its potential for broad applicability in advancing scientific research and applications.

AI4Mat Journal Track: Yes

Submission Number: 6