CF-Router: Closed-Form Solution for Expert Selection in Multimodal Agent Lifelong Learning

Jiaxu Li, Zhijie Zheng, Jianyu Qi, Rongchang Zhao

Keywords: Continual Learning; Multimodal Large Language Modal

TL;DR: CF-Router is a novel closed-form solution-based routing mechanism for multimodal agent lifelong learning.

Abstract: Multimodal Large Language Models (MLLMs) are increasingly pivotal as lifelong learning agents, tasked with adapting to evolving environments without succumbing to catastrophic forgetting. Current strategies often leverage Mixture-of-Experts (MoE) architectures combined with Low-Rank Adaptation (LoRA) to compartmentalize domain-specific knowledge. However, prevailing routing mechanisms—whether relying on MLLM prompting or heuristic similarity metrics—frequently suffer from low training efficiency or poor alignment within complex multimodal feature spaces. To address these limitations, we introduce $\textbf{CF-Router}$, a novel routing framework grounded in $\textbf{a closed-form solution}$. By leveraging the average-pooled hidden states from the MLLM's final layer as representative semantic descriptors, we employ a regularized least-squares classifier to precisely identify the optimal expert LoRA. This methodology facilitates analytic, mathematically optimal updates, guaranteeing rapid task identification and seamless adaptation for lifelong learning agents.

Submission Number: 18