HiCoLoRA: Addressing Context-Prompt Misalignment via Hierarchical Collaborative LoRA for Zero-Shot DST

ICLR 2026 Conference Submission12860 Authors

Keywords: Zero-shot, Dialogue State Tracking, Task-Oriented Dialog, LoRA, Transformers

TL;DR: HiCoLoRA introduces hierarchical LoRA, spectral clustering, and semantic-aware initialization to resolve context-prompt misalignment, achieving new SOTA in zero-shot dialog state tracking.

Abstract: Zero-shot Dialog State Tracking (zs-DST) is essential for enabling Task-Oriented Dialog Systems (TODs) to generalize to new domains without costly data annotation. A central challenge lies in the semantic misalignment between dynamic dialog contexts and static prompts, leading to inflexible cross-layer coordination, domain interference, and catastrophic forgetting. To tackle this, we propose Hierarchical Collaborative Low-Rank Adaptation (HiCoLoRA), a framework that enhances zero-shot slot inference through robust prompt alignment. It features a hierarchical LoRA architecture for dynamic layer-specific processing (combining lower-layer heuristic grouping and higher-layer full interaction), integrates Spectral Joint Domain-Slot Clustering to identify transferable associations (feeding an Adaptive Linear Fusion Mechanism), and employs Semantic-Enhanced SVD Initialization (SemSVD-Init) to preserve pre-trained knowledge. Experiments on multi-domain datasets MultiWOZ and SGD show that HiCoLoRA outperforms baselines, achieving SOTA in zs-DST. Code is available at \href{https://anonymous.4open.science/r/HiCoLoRA-96EB}{Anonymous Github}.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 12860