Disentangling Architecture and Training for Optical Flow

Deqing Sun, Charles Herrmann, Fitsum A. Reda, Michael Rubinstein, David J. Fleet, William T. Freeman

Abstract: How important are training details and datasets to recent optical flow architectures like RAFT? And do they generalize? To explore these questions, rather than develop a new architecture, we revisit three prominent architectures, PWC-Net, IRR-PWC and RAFT, with a common set of modern training techniques and datasets, and observe significant performance gains, demonstrating the importance and generality of these training details. Our newly trained PWC-Net and IRR-PWC show surprisingly large improvements, up to 30% versus original published results on Sintel and KITTI 2015 benchmarks. Our newly trained RAFT obtains an Fl-all score of 4.31% on KITTI 2015 and an avg. rank of 1.7 for end-point error on Middlebury. Our results demonstrate the benefits of separating the contributions of architectures, training techniques and datasets when analyzing performance gains of optical flow methods. Our source code is available at https://autoflow-google.github.io .