Motion-Guided Prior Support and Polarity Interaction for Event Stream Super-Resolution

Junchi Chen, Hongchen Tan, Xiuping Liu

Keywords: Event stream Super-resolution; Semantic Interaction and Fusion; Event Camera

Abstract:

In this paper, we aim to enhance the representation of spatio-temporal semantics during the Event stream Super-resolution (ESR) reconstruction by leveraging inter-frame motion information.
To this end, we propose a Motion-Guided Prior Support and Polarity Interaction Network (MPS-PI Net). The MPS-PI Net takes event frames as the primary input, while incorporating positive and negative event streams as auxiliary inputs. The MPS-PI Net contains two novel designs: Motion-Guided Semantic Prior (MGSP) Module and Bipolar Semantic Interaction and Fusion (B-SIF) Module. In the MGSP module, we capitalize on inter-frame optical flow information to seamlessly integrate semantic cues derived from previously reconstructed frames into the super-resolution reconstruction process of the current frame. This integration provides valuable prior support for reconstructing the content of the current frame with greater accuracy. Building upon the prior semantic information introduced by the MGSP module, within the B-SIF Module, we initially undertake self-representational enhancement for both positive and negative polarity semantics independently. Following this, we conduct an interactive fusion of these two polarity semantics to fully harness their unique advantages. Experimental results unequivocally demonstrate that our proposed MPS-PI Net achieves competitive performance on many ESR datasets.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 6637