Mitigating Surgical Data Imbalance with Dual-Prediction Video Diffusion Model

Danush Kumar Venkatesh, Adam Schmidt, Muhammad Abdullah Jamal, Omid Mohareri

Keywords: Synthetic data, Surgical Data Science, Data imbalance, Video diffusion

TL;DR: SurgiFlowVid: A dual-prediction diffusion framework using RGB and optical flow with sparse conditioning to address surgical video data imbalance, yielding 10–20% downstream performance gains.

Abstract: Surgical video datasets are essential for scene understanding, enabling procedural modeling and intra-operative support. However, these datasets are often heavily imbalanced, with rare actions and tools under-represented, which limits the robustness of downstream models. We address this challenge with SurgiFlowVid, a sparse and controllable video diffusion framework for generating surgical videos of under-represented classes. Our approach introduces a dual-prediction diffusion module that jointly denoises RGB frames and optical flow, providing temporal inductive biases to improve motion modeling from limited samples. In addition, a sparse visual encoder conditions the generation process on lightweight signals (e.g., sparse segmentation masks or RGB frames), enabling controllability without dense annotations. We validate our approach on three surgical datasets across tasks including action recognition, tool presence detection, and laparoscope motion prediction. Synthetic data generated by our method yields consistent gains of 10–20% over competitive baselines, establishing SurgiFlowVid as a promising strategy to mitigate data imbalance and advance surgical video understanding methods.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 5925