Can ChatGPT Solve Relation Extraction? An Extensive Assessment via Design Choice Exploration

Xinyi Wang, Wenzheng Zhao, Xiangrong Zhu, Wei Hu

Abstract: Large language models (LLMs), such as ChatGPT, have demonstrated remarkable abilities in simple information extraction tasks. However, when it comes to complex and demanding tasks like relation extraction (RE), LLMs may still have considerable space for improvement. In this paper, we extensively evaluate ChatGPT’s performance in RE to expose its strengths and limitations. We explore the design choices of ChatGPT’s input prompts for RE. Considering different combinations of these choices, we conduct thorough experiments on benchmark datasets and analyze ChatGPT’s comprehension abilities under different settings. Our experiment results provide insights for the future development of LLM-based RE models.