LCIF-Net: Local criss-cross attention based optical flow method using multi-scale image features and feature pyramid

Zige Wang, Zhen Chen, Congxuan Zhang, Zhongkai Zhou, Hao Chen

Abstract: Highlights•We design an image pyramid-based feature extraction sub-network, and incorporate it into the feature pyramid network to construct a hybrid feature extraction module. The presented feature extraction module combines both the semantic features and the textural and structural features to improve the accuracy of optical flow estimation, especially in the regions of image and motion boundaries.•We adopt the local criss-cross attention module to construct a global feature encoder network. The presented encoder network captures long-range dependencies from the long-distance pixels in the feature maps, which compensates the resulting optical flow and thus improves the overall performance in the regions of large displacements.•We evaluate our LCIF-Net method on MPI-Sintel and KITTI test datasets to conduct a comprehensive comparison with other state-of-the-art methods. The experimental results demonstrate that the proposed method performs competitive performance compared with the other methods, especially in regions of large displacements and motion boundaries.