CATS: cultural-heritage classification using LLMs and distribute model

Hyerin Hwang, Chan-Woo Park, Hee-Kwon KIM, Jae-Ho Lee

Abstract: In this paper, we introduce CATS (Cultural-heritage Advanced Translation Systems), a generative model applied to creating images for classification and exploring relationships among cultural heritage. We aimed to address the issue where large language models (LLMs) fail to generate appropriate sentences due to the limited training on classical Korean language, and the problem where text-to-image models trained on Korean language do not produce accurate sentences when using Korean words as they are. To solve this problem, a large language model was used to translate historical content containing classical Korean words into English sentences, which were then used as input for the text-to-image generation model. We found that the generation model using the translated English text produced more accurate and consistent images compared to the model using the original Korean text. Consequently, this approach offers highly convenient visual information for users and administrators at a low cost through the use of open-source models. Therefore, we propose the potential of a system that leverages generated images to facilitate the search and extraction of relevant information.