Learning to estimate optical flow using dual-frequency paradigm

Yujin Zheng, Chu He, Yan Huang, Shenghua Fan, Min Jiang, Dingwen Wang, Yang Yi

Abstract: Deep learning-based optical flow estimation achieved impressive success with faster inference time and outperformed performance. Optical flow estimation networks are usually treated as a black box relying on large amounts of synthetic data for training, therefore the generalization and robustness of the network applying in realities remains a challenge. To overcome these problems, a dual-frequency paradigm is proposed for optical flow estimation. The proposed dual-frequency encoder captures discriminative features with both high-frequency and low-frequency biases. It is experimentally demonstrated that our method achieves better generalization while only pre-trained on FlyingChiars. Furthermore, our method improves the prediction of optical flow in occluded regions by enhancing the perception of high-frequency features that further improve the robustness of the network. Compared to the start-of-the-art RAFT, our approach obtains an improvement of the average end-point error by 10.6% on the Sintel Clean datasets and 11.7% on the challenging Sintel Final dataset.