SURESTEP: An Uncertainty-Aware Trajectory Optimization Framework to Enhance Visual Tool Tracking for Robust Surgical Automation

Nikhil Uday Shinde, Zih-Yun Chiu, Florian Richter, Jason Lim, Yuheng Zhi, Sylvia Lee Herbert, Michael C. Yip

Abstract:

Inaccurate tool localization is one of the main reasons for failures in automating surgical tasks. Imprecise robot
kinematics and noisy observations caused by the poor visual
acuity of an endoscopic camera make tool tracking challenging.
Previous works in surgical automation adopt environmentspecific setups or hard-coded strategies instead of explicitly
considering motion and observation uncertainty of tool tracking in their policies. In this work, we present SURESTEP,
an uncertainty-aware trajectory optimization framework for
robust surgical automation. We model the uncertainty of tool
tracking with the components motivated by the sources of noise
in typical surgical scenes. Using a Gaussian assumption to
propagate our uncertainty models through a given tool trajectory, SURESTEP provides a general framework that minimizes
the upper bound on the entropy of the final estimated tool
distribution. We compare SURESTEP with a baseline method
on a real-world suture needle regrasping task under challenging
environmental conditions, such as poor lighting and a moving
endoscopic camera. The results over 60 regrasps on the da
Vinci Research Kit (dVRK) demonstrate that our optimized
trajectories significantly outperform the un-optimized baseline.