Towards Effective Surgical Representation Learning with DINO Models

Ronald L.P.D. de Jong, Yiping Li, Tim J. M. Jaspers, Romy C. van Jaarsveld, Gino M. Kuiper, Franco Badaloni, Richard van Hillegersberg, Jelle P. Ruurda, Fons van der Sommen, Josien P.W. Pluim, Marcel Breeuwer

Keywords: Anatomy recognition, DINO, Foundation models, Self-supervised learning, Surgical phase recognition

TL;DR: We show that DINOv1–v3 self-supervised models learn strong and scalable surgical representations across multiple datasets and settings.

Abstract: Self-supervised learning (SSL) has emerged as a promising approach to address the limitations of annotated surgical datasets, which are often small, heterogeneous, and expensive to curate. Among SSL methods, self-distillation with no labels (DINO) has achieved state-of-the-art (SOTA) results in natural images, but its applicability to surgical data remains underexplored. In this work, we systematically investigate DINOv1, DINOv2, and DINOv3 for surgical representation learning. We pretrain these models on a large-scale surgical dataset of 4.7M video frames (SurgeNetXL) and evaluate their transferability on downstream tasks including semantic segmentation and surgical phase recognition. Our results demonstrate that in-domain pretraining consistently improves performance across all DINO variants, with DINOv2 and DINOv3 achieving SOTA performance. We further offer practical insights and visualizations highlighting the effectiveness of SSL. Finally, our study delivers ready-to-use DINO-based SSL models and pretraining protocols for surgical AI research, which are publicly available at: https://github.com/rlpddejong/SurgeNetDINO.

Primary Subject Area: Unsupervised Learning and Representation Learning

Secondary Subject Area: Foundation Models

Registration Requirement: Yes

Reproducibility: https://github.com/rlpddejong/SurgeNetDINO

Visa & Travel: No

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Midl Latex Submission Checklist: Ensure no LaTeX errors during compilation., Created a single midl26_NNN.zip file with midl26_NNN.tex, midl26_NNN.bib, all necessary figures and files., Includes \documentclass{midl}, \jmlryear{2026}, \jmlrworkshop, \jmlrvolume, \editors, and correct \bibliography command., Did not override options of the hyperref package, Did not use the times package., All authors and co-authors are correctly listed with proper spelling and avoid Unicode characters., Author and institution details are de-anonymized where needed. All author names, affiliations, and paper title are correctly spelled and capitalized in the biography section., References must use the .bib file. Did not override the bibliographystyle defined in midl.cls. Did not use \begin{thebibliography} directly to insert references., Tables and figures do not overflow margins; avoid using \scalebox; used \resizebox when needed., Included all necessary figures and removed *unused* files in the zip archive., Removed special formatting, visual annotations, and highlights used during rebuttal., All special characters in the paper and .bib file use LaTeX commands (e.g., \'e for é)., Appendices and supplementary material are included in the same PDF after references., Main paper does not exceed 12 pages; acknowledgements, references, and appendix start on page 13 or later.

Submission Number: 83