3D Key-Points Estimation from Single-View RGB Images

Mohammad Zohaib, Matteo Taiana, Milind Gajanan Padalkar, Alessio Del Bue

Abstract: The paper presents an end-to-end approach that leverages images for estimating an ordered list of 3D key-points. Most of the existing methods either use point clouds or multiple RGB/depth images to estimate 3D key-points, whereas the proposed approach requires only a single-view RGB image. It is based on three steps: extracting latent codes, computing pixel-wise features, and estimating 3D key-points. It also computes a confidence score of every key-point that enables it to predict a different number of key-points based on an object’s shape. Therefore, unlike existing approaches, the network can be trained to address several categories at once. For evaluation, we first estimate 3D key-points for two views of an object and then use them for finding a relative pose between the views. The results show that the average angular distance error of our approach (6.39 $$^{\circ }$$ ) is 8.01 $$^{\circ }$$ lower than that of KP-Net (14.40 $$^{\circ }$$ ) [1].