Distil-E2D: Distilling Image-to-Depth Priors for Event-Based Monocular Depth Estimation

Jie Long Lee, Gim Hee Lee

Keywords: event camera, neuromorphic vision, monocular depth estimation

TL;DR: We propose Distil-E2D for event-based monocular depth estimation, using dense synthetic depths from foundational models, a confidence-guided calibrated loss for label alignment, and a novel architecture to improve encoder representation.

Abstract: Event cameras are neuromorphic vision sensors that asynchronously capture pixel-level intensity changes with high temporal resolution and dynamic range. These make them well suited for monocular depth estimation under challenging lighting conditions. However, progress in event-based monocular depth estimation remains constrained by the quality of supervision: LiDAR-based depth labels are inherently sparse, spatially incomplete, and prone to artifacts. Consequently, these signals are suboptimal for learning dense depth from sparse events. To address this problem, we propose Distil-E2D, a framework that distills depth priors from the image domain into the event domain by generating dense synthetic pseudolabels from co-recorded APS or RGB frames using foundational depth models. These pseudolabels complement sparse LiDAR depths with dense semantically rich supervision informed by large-scale image-depth datasets. To reconcile discrepancies between synthetic and real depths, we introduce a Confidence-Guided Calibrated Depth Loss that learns nonlinear depth alignment and adaptively weights supervision by alignment confidence. Additionally, our architecture integrates past predictions via a Context Transformer and employs a Dual-Decoder Training scheme that enhances encoder representations by jointly learning metric and relative depth abstractions. Experiments on benchmark datasets show that Distil-E2D achieves state-of-the-art performance in event-based monocular depth estimation across both event-only and event+APS settings.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 9171