RecResNet: A Recurrent Residual CNN Architecture for Disparity Map Enhancement

Konstantinos Batsos, Philippos Mordohai

Abstract:

We present a neural network architecture applied to the
problem of refining a dense disparity map generated by
a stereo algorithm to which we have no access. Our approach is able to learn which disparity values should be
modified and how, from a training set of images, estimated
disparity maps and the corresponding ground truth. Its only
input at test time is a disparity map and the reference image. Two design characteristics are critical for the success
of our network: (i) it is formulated as a recurrent neural
network, and (ii) it estimates the output refined disparity
map as a combination of residuals computed at multiple
scales, that is at different up-sampling and down-sampling
rates. The first property allows the network, which we named
RecResNet, to progressively improve the disparity map, while
the second property allows the corrections to come from
different scales of analysis, addressing different types of
errors in the current disparity map. We present competitive quantitative and qualitative results on the KITTI 2012
and 2015 benchmarks that surpass the accuracy of previous disparity refinement methods. Our code is available at
https://github.com/kbatsos/RecResNet