Predicting High-precision Depth on Low-Precision Devices Using 2D Hilbert Curves

Mykhail Uss, Ruslan Yermolenko, Oleksii Shashko, Olena Kolodiazhna, Ivan Safonov, Volodymyr Savin, Yoonjae Yeo, Seowon Ji, Jaeyun Jeong

TL;DR: High-precision depth prediction on low-end devices with low-precision calculations using depth representation as two 2D Hilbert curve components

Abstract: Dense depth prediction deep neural networks (DNN) have achieved impressive results for both monocular and binocular data but they are limited by high computational complexity, restricting their use on low-end devices. For better on-device efficiency and hardware utilization, weights and activations of the DNN should be converted to low-bit precision. However, this precision is not sufficient for representing high dynamic range depth. In this paper, we aim to overcome this limitation and restore high-precision depth from low-bit precision predictions. To achieve this, we propose to represent high dynamic range depth as two low dynamic range components of a Hilbert curve, and to train the full precision DNN to directly predict the latter. For on-device deployment, we use standard quantization methods and add a post-processing step that reconstructs depth from the Hilbert curve components predicted in low-bit precision. Extensive experiments demonstrate that our method increases bit precision of predicted depth by up to three bits with little computational overhead. We also observe a positive side effect of quantization error reduction by up to five times. Our method enables effective and accurate depth prediction with DNN weights and activations quantized to eight bit precision.

Lay Summary:

We use depth prediction to help machines understand and interact with their surroundings in such areas as augmented reality, robotics, and autonomous vehicles. These applications often rely on devices with limited computing power. Deep neural networks (DNNs) are powerful tools for depth prediction but require significant processing resources, which low-end devices struggle to provide. To address this, DNN’s are converted to a lower precision, reducing its demands on hardware. Unfortunately, this conversion often compromises the accuracy of depth predictions.

Our task was to overcome this limitation and restore high-precision depth from low-precision predictions. We took a novel approach: breaking down complex depth information into simpler parts using a transformation based on special Hilbert curves. By training the DNN to predict these simpler parts, we could later predict them on device in low precision an reconstruct the full depth with higher precision. Extensive experiments demonstrated that our method increases the quality of details in depth predicted on device to a level comparable with the original full precision model. This innovation makes it possible for DNNs to deliver accurate depth predictions even when their calculations are simplified for low-end devices, paving the way for more accessible and effective technology in everyday applications.

Primary Area: Applications->Computer Vision

Keywords: dense depth prediction, depth-from-stereo, on-device deployment, low-end devices, low-precision computations, high dynamic range, hilbert curve, quantization

Submission Number: 11542