MEDS Decentralized, Extensible Validation (MEDS-DEV) Benchmark: Establishing Reproducibility and Comparability in ML for Health

Matthew B.A. McDermott, Aleksia Kolo, Chao Pang, Edward Choi, Ethan Steinberg, Hyewon Jeong, Jack Gallifant, Jason Alan Fries, Jeffrey N Chiang, Jungwoo Oh, Justin Xu, Kamilė Stankevičiūtė, Kiril Vadimovic Klein, Mikkel Fruelund Odgaard, Nassim Oufattole, Patrick Rockenschaub, Pawel Renc, Robin van de Water, Shalmali Joshi, Simon Austin Lee et al. (11 additional authors not shown)

Abstract: We introduce the Medical Event Data Standard (MEDS), a lightweight schema for enabling machine learning over electronic health record (EHR) data. Unlike common data models and data interoperability formats, MEDS is a minimal standard designed for maximum interoperability across datasets, existing tools, and model architectures. By providing a simple standardization layer between datasets and model-specific code, MEDS will enable more reproducible, robust, computationally performant, and collaborative machine learning research using EHR data. We highlight several existing MEDS integrations with models, datasets, and tools, and invite the community for further development and adoption.