Developing and Testing a New Machine-Learning Method to Identify Patients with Heart Failure Who Are at Risk of 30-Day Readmission or Mortality

Background: Predicting readmissions or mortality following hospital discharge in patients with heart failure (HF) is a major challenge. Commonly used machine learning (ML) algorithms have reported suboptimal performances in predicting HF readmissions or mortality due to class imbalance in the outcome levels (e.g. low number of readmissions vs no readmission). Moreover, the area under the receiver operating characteristic curve (AUC), a commonly used indicator of model performance, is a crude indicator for data with class imbalances. Our main aim was to develop ML models to predict 30-day HF readmission or mortality and compare various measures of model performance. Methods: We identified all Western Australian patients 65 years or older admitted for HF in 2003–08 in the linked Hospital Morbidity Data Collection. In addition to common ML models to predict 30-day HF readmissions or mortality, we also developed multi-layer perceptron (MLP) with a synthetic minority oversampling technique, considered superior in data with class imbalances. We calculated the AUC, sensitivity, and specificity of the models. Results: Of the 10,735 patients with HF, 23.6% were readmitted or died within 30 days. We observed AUCs of 0.51, 0.50, and 0.51 for linear regression, least absolute shrinkage and selection operator, and random forest models, respectively. The MLP model had a higher AUC (0.58), with a sensitivity of 57% and a specificity of 56%. Conclusion: The MLP method may be a better ML technique for data with outcome class imbalances. In addition to AUC, other metrics such as sensitivity and specificity should also be considered for such data.