Multi-Class Bayesian Segmentation of Robotically Acquired Ultrasound Enabling 3D Site Selection along Femoral Vessels for Planning Safer Needle Insertion

Edward Chen, FNU Abhimanyu, Ananya Bal, Nishanth Thumbavanam Arun, Andrew Schoenling, Joo Yoon, Kelvin Kwofie, Lenny Weiss, Marek Radomski, Frank Guyette, Howie Choset, John Galeotti

Keywords: Multi-class Segmentation, 3D Reconstruction, Needle Insertion, POCUS AI

TL;DR: We perform Bayesian Segmentation and 3D Reconstruction for Safe Needle Insertion

Abstract: We present the first known system to use robotic ultrasound to accurately map multiple anatomic structures in the femoral region in 3D and choose a point for vascular needle insertion that is suitable for catheter placement. The maps are presented as three-dimensional point clouds with points labeled categorically, e.g., veins, arteries, ligaments, etc. The final point for insertion is presented on a 3D map. We use a multi-class, multi-instance Bayesian 3D Convolutional Neural Network (CNN) to segment and identify the anatomic structures from 2D time series ultrasound data. The 2D segmented slices are then temporally stacked and synced with the kinematics of the robot maneuvering the ultrasound probe to create a 3D point cloud. This 3D point cloud is analyzed based on heuristics from physicians, to determine an ideal point to puncture with the needle, i.e., solve the needle-insertion planning problem. In particular, we determine the desired insertion point in either the common femoral artery or vein. We achieved a Jaccard score J = 0.834 for vessel segmentation and were able to determine safe insertion points in 46 out of 49 trials. Our system requires minimal human intervention and is designed to be robust to changes in ultrasound imaging settings and subject anatomy.

Registration: I acknowledge that publication of this at MIDL and in the proceedings requires at least one of the authors to register and present the work during the conference.

Authorship: I confirm that I am the author of this work and that it has not been submitted to another publication before.

Paper Type: both

Primary Subject Area: Segmentation

Secondary Subject Area: Image Acquisition and Reconstruction

Confidentiality And Author Instructions: I read the call for papers and author instructions. I acknowledge that exceeding the page limit and/or altering the latex template can result in desk rejection.