Real-Time Ultrasound Image Synthesis for Deformable Soft Tissue Simulation in Surgery

Yafei Ou, Mahdi Tavakoli

Keywords: Surgical simulation, Ultrasound imaging, Ultrasound simulation

Abstract: A number of recent studies in surgical robotics have focused on developing realistic surgical simulation environments or platforms that can be used to train surgical AI agents, especially through reinforcement learning (RL) or imitation learning (IL). However, the focus so far has been primarily on simulating visually plausible scenes in terms of the RGB image observation, such as how the endoscopic camera image looks when tissue deforms or is burnt after cauterization. Other medical imaging modalities, such as ultrasound (US) imaging, are frequently used during surgeries, which may also be used as observations for surgical AI agents. Therefore, simulating such images is an important step towards training agents that can utilize other imaging modalities. This work proposes a real-time US image synthesis method for surgical soft body under randomized deformation.

Submission Number: 35