EgoExo-Con: Exploring View-Invariant Video Temporal Understanding

ICLR 2026 Conference Submission4057 Authors

Keywords: Video Large Language Models;

TL;DR: We introduce EgoExo-Con, benchmarking view-invariant temporal understanding in Video-LLMs. We reveal that viewpoint variation remains a significant challenge for Video-LLMs and propose View-GRPO for improvement.

Abstract: Can Video-LLMs achieve consistent temporal understanding when videos capture the same event from different viewpoints? To study this, we introduce EgoExo-Con (Consistency), a benchmark of comprehensively synchronized egocentric and exocentric video pairs with human-refined queries in natural language. EgoExo-Con emphasizes two temporal understanding tasks: Temporal Verification and Temporal Grounding. It evaluates not only correctness but consistency across viewpoints. Our analysis reveals two critical limitations of existing Video-LLMs: (1) models often fail to maintain consistency, with results far worse than their single-view performances. (2) When naively finetuned with synchronized videos of both viewpoints, the models show improved consistency but often underperform those trained on a single view. For improvements, we propose View-GRPO, a novel reinforcement learning framework that effectively strengthens view-specific temporal reasoning while encouraging consistent comprehension across viewpoints. Our method demonstrates its superiority over naive SFT and GRPO, especially for improving cross-view consistency. All resources will be made publicly available.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 4057