LoCar: Localization-Aware Evaluation of In-Vehicle Assistants through Fine-Grained Sociolinguistic Control

Seogyeong Jeong, Kiwoong Park, Seyoung Song, Eunsu Kim, Ken E Friedl, Jaeho Kim, Alice Oh

Keywords: In-Vehicle Assistant, LLM Evaluation Framework, Localization, Korean Honorifics, Car LLM

TL;DR: LoCar: A localization-aware framework for in-vehicle AI that ensures precise control of Korean honorifics and strategic interaction reliability to meet rigorous sociolinguistic and brand consistency standards.

Abstract: While Large Language Models (LLMs) are increasingly integrated into in-vehicle conversational systems, identifying the optimal model remains challenging due to the lack of domain-specific evaluation standards tailored to real-world deployment requirements. In this paper, we propose a novel evaluation framework for in-vehicle assistants, with a particular focus on Korean-language localization. Our empirical analysis reveals notable patterns in model behavior. First, fine-grained Korean honorific control remains unstable in current LLMs, indicating that precise speech-level realization must be explicitly evaluated in localization settings. Second, models exhibit weaker performance in strategic conversational metrics like clarification and proactivity. Our analysis suggests this stems from the inherent subjective complexity of these tasks, where our framework adopts a conservative evaluation stance to prioritize reliability. Together, our findings underscore that automotive AI must move beyond general competence toward precise linguistic tailoring and reliable, safety-oriented interaction management.

Submission Type: Emerging

Submission Number: 523