Minibatches can make neural network training repeatable for clinical applications

Alex Breuer, Larissa V. Furtado, Brent A. Orr

Confirmation: I have read and agree with the IEEE BHI 2025 conference submission's policy on behalf of myself and my co-authors.

Keywords: neural networks, methylation classification of tumors, clinical classifier deployment, numerical analysis

TL;DR: Round-off error can cause repeatability problems for clinical classifiers with long lifecycles, minibatched training can eliminate this problem.

Abstract:

To qualify for clinical use, an AI classifier must first satisfy a set
of requirements to ensure its safety and efficacy. We concentrate on
longitudinal consistency over its deployment lifecycle, which
may be summarized as: different builds or versions of the classifier
must not conclusively produce discrepant results. This requirement is
important when a classifier is used for serial monitoring or for
stratification for a clinical trial. Our main contribution is an
analysis that explains how round-off error can cause training
repeatability issues bad enough to create difficulties for
longitudinal consistency, and how these training issues may be remediated
by using minibatches when training. This analysis is based on a
simple neural network that fails to satisfy longitudinal consistency
when trained on different hardware due to round-off error that is not
controlled by PyTorch's deterministic training mode, even when
all software, training hyperparameters and all other initial
conditions are held fixed, computation is done in double precision,
and deterministic training is used. Our example further shows that
minibatched training remedies this longitudinal repeatability problem.
These results inform the R&D and test and evaluation procedures that should be used
for clinical-grade classifiers.

Track: 4. Clinical Informatics

Registration Id: 7CNFGW2PKB2

Submission Number: 293