Cardiac Operative Risk in Latin America: A Comparison of Machine Learning Models vs EuroSCORE-II

Raul Molina, Maria Alejandra Molina-Rodriguez

Abstract:

BACKGROUND Machine learning is a useful tool for predicting medical outcomes. This study aimed to develop a
machine learning–based preoperative score to predict cardiac surgical operative mortality.
METHODS We developed various models to predict cardiac operative mortality using machine learning techniques and
compared each model to European System for Cardiac Operative Risk Evaluation-II (EuroSCORE-II) using the area
under the receiver operating characteristic (ROC) and precision-recall (PR) curves (ROC AUC and PR AUC) as perfor-
mance metrics. The model calibration in our population was also reported with all models and in high-risk groups for
gradient boosting and EuroSCORE-II. This study is a retrospective cohort based on a prospectively collected database
from July 2008 to April 2018 from a single cardiac surgical center in Bogota, Colombia.
RESULTS Model comparison consisted of hold-out validation: 80% of the data were used for model training, and the
remaining 20% of the data were used to test each model and EuroSCORE-II. Operative mortality was 6.45% in the entire
database and 6.59% in the test set. The performance metrics for the best machine learning model, gradient boosting
(ROC: 0.755; PR: 0.292), were higher than those of EuroSCORE-II (ROC: 0.716, PR: 0.179), with a P value of .318 for the
AUC of the ROC and .137 for the AUC of the PR.
CONCLUSIONS The gradient boosting model was more precise than EuroSCORE-II in predicting mortality in our
population based on ROC and PR analyses, although the difference was not statistically significant.