BEAMRAD: A tool for Creating and Assessing Medical Dataset Documentation

Maria Galanty, Dieuwertje Luitse, Alexander P. Vlaar, Clara I. Sánchez, Tobias Blanke, Ivana Isgum

Keywords: Medical datasets, Dataset documentation, Bias

TL;DR: To address bias stemming from dataset documentation practices, we introduce BEAMRAD, a tool to create and assess medical dataset documentation.

Abstract: Medical datasets drive deep learning in medical imaging but may introduce biases that impact model performance and clinical applicability. To address these bias challenges, we introduce BEAMRAD, a dynamic tool to create and assess medical dataset documentation. BEAMRAD systematically evaluates documentation, and links insufficient reporting to potential biases. Through an exemplary assessment of publicly available medical datasets, we highlight gaps in dataset documentation, including inconsistencies in data annotation, error quantification, and dataset limitations reporting. We propose to address these issues with three key improvements: stricter repository oversight, reflective documentation practices, and adaptable documentation.

Submission Number: 29