Differentiable rendering-based pose estimation for surgical robotic instruments

Zekai Liang, Zih-Yun Chiu, Florian Richter, Michael C. Yip

Abstract:

Robot pose estimation is a challenging and crucial task for vision-based surgical robotic automation. Typical
robotic calibration approaches, however, are not applicable to surgical robots, such as the da Vinci Research Kit (dVRK), due to joint angle measurement errors from cable-drives and
the partially visible kinematic chain. Hence, previous works in
surgical robotic automation used tracking algorithms to estimate the pose of the surgical tool in real-time and compensate
for the joint angle errors. However, a big limitation of these
previous tracking works is the initialization step which relied
on only keypoints and SolvePnP. In this work, we fully explore
the potential of geometric primitives beyond just keypoints with
differentiable rendering, cylinders, and construct a versatile
pose matching pipeline in a novel pose hypothesis space. We
demonstrate the state-of-the-art performance of our single-shot
calibration method with both calibration consistency and real
surgical tasks. As a result, this marker-less calibration approach
proves to be a robust and generalizable initialization step for
surgical tool tracking.