MoRA: Missing Modality Low-Rank Adaptation for Visual Recognition

ICLR 2026 Conference Submission20667 Authors

Keywords: Multimodal Classification, Missing Modality, Parameter-Efficient Fine-Tuning

TL;DR: MoRA is a PEFT method for multimodal models that handles missing modalities during inference, resulting in better performance, faster inference, and minimal trainable parameters compared to existing approaches

Abstract: Pre-trained vision language models have shown remarkable performance on visual recognition tasks, but they typically assume the availability of complete multimodal inputs during both training and inference. In real-world scenarios, however, modalities may be missing due to privacy constraints, collection difficulties, or resource limitations. While previous approaches have addressed this challenge using prompt learning techniques, they fail to capture the cross-modal relationships necessary for effective multimodal visual recognition and suffer from inevitable computational overhead. In this paper, we introduce MoRA, a parameter-efficient fine-tuning method that explicitly models cross-modal interactions while maintaining modality-specific adaptations. MoRA introduces modality-common parameters between text and vision encoders, enabling bidirectional knowledge transfer. Additionally, combined with the modality-specific parameters, MoRA allows the backbone model to maintain inter-modality interaction and enable intra-modality flexibility. Extensive experiments on standard benchmarks demonstrate that MoRA achieves an average performance improvement in missing-modality scenarios by 5.24% and uses only 25.90% of the inference time compared to the SOTA method while requiring only 0.11% of trainable parameters compared to full fine-tuning. The code is available at https://anonymous.4open.science/r/mora-20667.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 20667