Real-time optical flow estimation on vein and artery ultrasound sequences based on knowledge-distillation

Till Nicke, Laura Graf, Mikko Lauri, Sven Mischkewitz, Simone Frintrop, Mattias P Heinrich

Keywords: knowledge compression, realtime video inference, ultrasound images

TL;DR: In this Paper we showcase the impact of cross domain knowledge compression on a lightweight optical flow estimator for real time inference on ultrasound videos

Abstract:

In this paper, we propose an approach for realtime optical flow estimation in ultrasound sequences of vein and arteries based on knowledge
distillation. Knowledge distillation is a technique to train a faster, smaller model by learning from cues of other models. Mobile devices with limited resources could be key in providing effective point-of-care healthcare and motivate the search of more lightweight solutions in the deep learning based image analysis. For ultrasound video analysis motion correspondences of image contents (anatomies) have to be computed for temporal context and for real time application, fast solutions are required. We use a PWC-Net's optical flow estimation output as soft targets to train a lightweight optical flow estimator. We analyse how well it works on the challenging task of fast segmentation propagation of vein and arteries in ultrasound images. Experiments show that even though we did not fine-tune the teachers on this task, a model trained with soft targets outperformed a model trained directly with labels and without a teacher.

Dataset Code: https://github.com/TillNicke/cross-domain-kc (dataset not shareable in the next few years since it is property of the collaborating company)