Asymmetric Event-Guided Video Super-Resolution
Abstract: Event cameras are novel bio-inspired cameras that record asynchronous events with high temporal resolution and dynamic range. Leveraging the auxiliary temporal information recorded by event cameras holds great promise for the task of video super-resolution (VSR). However, existing event-guided VSR methods assume that the event and RGB cameras are strictly calibrated (e.g., pixel-level sensor designs in DAVIS 240/346). This assumption proves limiting in emerging high-resolution devices, such as dual-lens smartphones and unmanned aerial vehicles, where such precise calibration is typically unavailable. To unlock more event-guided application scenarios, we propose to perform the task of asymmetric event-guided VSR for the first time, and we propose an Asymmetric Event-guided VSR Network (AsEVSRN) for this new task. AsEVSRN incorporates two specialized designs for leveraging the asymmetric event stream in VSR. Firstly, the content hallucination module dynamically enhances event and RGB information by exploiting their complementary nature, thereby adaptively boosting representational capacity. Secondly, the event-enhanced bidirectional recurrent cells align and propagate temporal features fused with features from content-hallucinated frames. Within the bidirectional recurrent cells, event-enhanced flow is employed for simultaneous utilization and fusion of temporal information at both the feature and pixel levels. Comprehensive experimental results affirm that our method consistently produces superior results both quantitatively and qualitatively. Code will be released.
Relevance To Conference: High-resolution (HR) videos are attracting increasing attention in both academia and industry, and have been already widely used in modern society, especially for the multimedia field. Video super-resolution (VSR) stands as a foundational task within the domains of computer vision to generate HR videos. In this paper, we address the challenge of performing asymmetric event-guided VSR for the first time, introducing an AsEVSRN tailored for this novel task.
Supplementary Material: zip
Primary Subject Area: [Experience] Multimedia Applications
Secondary Subject Area: [Content] Media Interpretation
Submission Number: 3531
Loading