CUTDEM: Depth-Aware Enhanced Multi-View Image Mixing for Light Field Super-Resolution

Ze-Yu Mi, Yu-Bin Yang

Abstract: Data Augmentation (DA) is a powerful technique for enhancing the generalization capabilities of deep neural networks. It has been successfully employed in super-resolution research, resulting in improved performance. However, current DA methods often struggle to adapt effectively to light field super-resolution tasks due to their limited utilization of multi-view information. In this paper, we conduct a comprehensive analysis of DA techniques as applied to sub-aperture images in light field super-resolution. Our investigation reveals that depth information can provide valuable insights from different perspectives. Building upon our findings, we introduce a novel DA approach named CutDEM, which harnesses multi-view information more efficiently and is compatible with various network architectures. Specifically, CutDEM first extracts low-resolution (LR) patches from different viewpoints, aligning them at corresponding positions, and subsequently calculates depth information based on texture characteristics. The depth information is then transformed into weight coefficients for each view image, facilitating the mixing of these patches. Finally, the mixed patches are seamlessly integrated into high-resolution (HR) images within their respective regions. Our experimental results demonstrate that CutDEM significantly enhances the utilization of multi-view information in DA strategies and consistently improves performance across different existing light field super-resolution networks.