Believing in black boxes: machine learning for healthcare does not need explainability to be evidence-based

Liam G. McCoy, Karina Vold

Abstract:

Objective: To examine the role of explainability in machine learning for healthcare (MLHC), and its necessity and significance with
respect to effective and ethical MLHC application.
Study Design and Setting: This commentary engages with the growing and dynamic corpus of literature on the use of MLHC and
artificial intelligence (AI) in medicine, which provide the context for a focused narrative review of arguments presented in favour of
and opposition to explainability in MLHC.
Results: We find that concerns regarding explainability are not limited to MLHC, but rather extend to numerous well-validated
treatment interventions as well as to human clinical judgment itself. We examine the role of evidence-based medicine in evaluating
inexplicable treatments and technologies, and highlight the analogy between the concept of explainability in MLHC and the related
concept of mechanistic reasoning in evidence-based medicine.
Conclusion: Ultimately, we conclude that the value of explainability in MLHC is not intrinsic, but is instead instrumental to
achieving greater imperatives such as performance and trust. We caution against the uncompromising pursuit of explainability, and
advocate instead for the development of robust empirical methods to successfully evaluate increasingly inexplicable algorithmic systems.