Go to ICLR 2026 Conference homepageDimensional Debiasing via Multi-Agent Correction
Keywords: Multi-Agent System, Vison Bias, Multimodal LLMs
TL;DR: Multimodal LLMs often take shortcuts from biased data (such as always guessing a clock shows 10:10). We built a novel Multi-Agent Debiasing Correction, to identify and correct these biased responses.
Abstract: Multimodal Large Language Models (MLLMs) recognize patterns from diverse data dimensions, such as shape, color, and associated language cues. However, inherent biases in training data can lead MLLMs to learn unintended, harmful shortcuts. For example, MLLMs often misinterpret clock times as defaulting to 10:10 due to memorized visual patterns rather than analyzing clock-hand positions. To address this, we propose the Multi-Agent Debiasing (MAD) framework, which performs cross-dimensional verification to correct these shortcut-driven errors. We first derive six dimensions of debiasing guidelines through a systematic analysis of failure responses. These guidelines inform the design of a team of specialized "dimension critic" agents, each an expert in correcting a specific type of error related to either biased cognition or limited perception. In our framework, potentially biased responses are dynamically routed through relevant agents. They then refine and correct the response in cascade over subsequent rounds. We leverage this cascaded correction process as a data engine to build our Multi-Dimensional Debiasing Dataset (MD$^3$), a large-scale collection of rich, debiased reasoning chains. By fine-tuning a model on MD$^3$, we directly teach it to overcome shortcut learning. Our experiments show that the MAD process encourages deeper thinking on biased responses. The MAD framework proves highly effective in classical visual debiasing settings and significantly enhances the reliability of MLLMs.
Primary Area: applications to computer vision, audio, language, and other modalities
Submission Number: 4628
Loading