High Accuracy TOF and Stereo Sensor Fusion at Interactive Rates

Rahul Nair, Frank Lenzen, Stephan Meister, Henrik Schäfer, Christoph S. Garbe, Daniel Kondermann

Abstract: We propose two new GPU-based sensor fusion approaches for time of flight (TOF) and stereo depth data. Data fidelity measures are defined to deal with the fundamental limitations of both techniques alone. Our algorithms combine TOF and stereo, yielding megapixel depth maps, enabling our approach to be used in a movie production scenario. Our local model works at interactive rates but yields noisier results, whereas our variational technique is more robust at a higher computational cost. The results show an improvement over each individual method with TOF interreflection remaining an open challenge. To encourage quantitative evaluations, a ground truth dataset is made publicly available.