Occlusion-Free 4D Gaussians for Open Surgery Videos Using Multi-camera Shadowless Lamps

Yuna Kato, Shohei Mori, Hideo Saito, Yoshifumi Takatsume, Hiroki Kajita, Mariko Isogawa

Abstract: Video recording of open surgery is in great demand for education and research purposes but is challenging due to the busy and dynamic environment. The state-of-the-art system uses multi-view cameras installed in shadowless lamps (McSL) and implements an automatic camera switching algorithm to avoid disturbances. However, this algorithm leads to missing pixels and distorted projection due to mathematical image warping and does not always provide the best perspective. We propose using 4D Gaussian Splatting (4DGS) to create editable 3D videos and remove Gaussians occluding surgical fields from a perspective. We enable occlusion-free 3D videos by addressing two occlusion removal approaches via (1) occlusion masking and (2) density-based Gaussian filtering. We create a real-surgery dataset and demonstrate that our method outperforms the state-of-the-art auto view-switching approach.

External IDs:dblp:conf/miccai/KatoMSTKI25