Machine learning-based model for predicting major adverse cardiovascular and cerebrovascular events in patients aged 65 years and older undergoing noncardiac surgery

Abstract

Background

Few evidence-based prediction models have been developed for predicting major adverse cardiovascular and cerebrovascular events (MACCE) in patients aged 65 years or older undergoing noncardiac surgery. In this study, we aimed to analyze the risk factors for perioperative MACCE in patients aged 65 years or older undergoing noncardiac surgery and construct a prediction model.

Methods

In this nested case–control study, a total of 342 Chinese patients who were aged ≥ 65 years and underwent medium- or high-risk noncardiac surgery in our hospital were included. There were 84 cases with MACCE (the MACCE group) and 258 without MACCE (the control group). Univariable logistic regression analysis was performed to identify the risk factors for MACCE. Least absolute shrinkage and selection operator (LASSO) regression was used to screen the variables. Nomogram was constructed using the selected variables. Machine learning methods, including Decision Tree, XGBoost, Support Vector Machine, K-nearest Neighbor, and Neural network, was used to establish, validate, and compare the performance of different prediction models.

Results

A prediction model based on nine variables, including age ≥ 85 years, history of ischemic chest pain, symptoms of decompensated heart failure, high-risk surgery, intraoperative minimum systolic blood pressure, postoperative systolic blood pressure, Cr levels over 2.0 mg/dL, left ventricular ejection fraction, and perioperative blood transfusion, was constructed. This LASSO logistic regression model showed good discriminatory ability to predict MACCE (area under the curve = 0.89; 95% confidence interval, 0.818 – 0.963) and fit to the test set (Hosmer–Lemeshow, χ2 = 7.4053, P = 0.4936). The decision curve analysis showed a positive net benefit of the new model. Compared with logistic regression model, the XGBoost model showed better prediction ability (area under the curve = 0.903). A preoperative prediction model based on five variables, including age ≥ 85 years, symptoms of decompensated heart failure, ischemic chest pain, high-risk type of surgery and Cr levels over 2.0 mg/dL was also constructed. This model showed good discriminatory ability to predict MACCE before surgery (area under the curve = 0.720 [95% CI, 0.591–0.848]. Both models compared with the modified RCRI score had improvement in reclassification.

Conclusion

By analyzing Chinese patients aged ≥ 65 years undergoing medium- or high-risk noncardiac surgery, the risk factors for perioperative MACCE were identified. Then, simple prediction models were constructed and validated, which showed good prediction performance and may be used as a decision-making assistant tool for clinicians. These findings provide a basis for preventing and improving the perioperative management of MACCE.

Background

With the increase in the aging population and the advancement in medical treatment, more elderly patients have received noncardiac surgery. However, previous studies and some guidelines have suggested that advanced age is an independent risk factor for major adverse cardiovascular [1,2,3] and cerebrovascular events [1, 4, 5] (MACCE) during the perioperative period of noncardiac surgery. The prevalence of comorbid conditions and the surgical types differ significantly between young and elderly patients [6]. The validated preoperative risk scoring system can be used to predict the risk of adverse events in patients undergoing noncardiac surgery. At present, three scoring systems for evaluating preoperative cardiac risk have been recognized in the guidelines: the modified Revised Cardiac Risk Index (RCRI), [7] the National Surgical Quality Improvement Program (NSQIP) Myocardial Infarction & Cardiac Arrest (MICA) calculator, [8] and the American College of Surgeons NSQIP (ACS NSQIP) Surgical Risk Calculator [9]. First of all, no currently available cardiac risk evaluation tools are designed for elderly patients. Secondly, the incidence rate of cerebrovascular events after non cardiac surgery is increasing significantly, but none of the three cardiac risk evaluation tools include cerebrovascular diseases as endpoints [10]. Thirdly,with the improvement in surgical methods and medical care in recent years, cardiac risk evaluation tools that were developed a long time ago need to be updated to meet the current clinical needs. -Finally, most cardiac risk evaluation tools have not been validated in the Chinese population. Therefore, it is of great significance to establish a model to predict the risk of MACCE during the perioperative period of noncardiac surgery for elderly Chinese patients.

In this study, we verified the effectiveness of the modified RCRI score recommended by the guidelines in the elderly population of China. Then we identified the risk factors for perioperative MACCE in elderly patients undergoing noncardiac surgery. We further established and validated new prediction models,and different prediction models were constructed and compared by machine learning methods. Lastly, we compared the new prediction models with modified RCRI score.

Methods

Ethical statement

This study was approved by the Ethics Committee of the Beijing Chaoyang Hospital (approval no. 2021-S-476). Informed consent was waived because of the retrospective nature of the study. All data were anonymous.

Population and outcomes

All elective and elevated-risk surgeries requiring anesthesia that were performed at the Beijing Chaoyang Hospital between January 1st, 2018, and June 30th, 2022, were screened. According to the 2014 ESC/ESA Guidelines on non-cardiac surgery, elevated-risk surgery includes intermediate surgery (i.e., intraperitoneal surgery, carotid endarterectomy, peripheral arterioplasty, endovascular aneurysm repair, head and neck surgery, neurosurgery/major orthopedic surgery, neurosurgery/major orthopedic surgery, and major urological surgery) and high-risk surgery (i.e., aortic and other major vascular surgery, and peripheral vascular surgery) [3]. If the patient underwent more than one surgery in 4 months, only the first one was included. Patients were excluded if they (1) underwent transplantation or had traumatic injury; (2) were aged less than 65 years; (3) were at the American Society of Anesthesiologists Classification V; (4) underwent palliative surgery of advanced malignant tumor; (5) had congenital heart disease; (6) underwent cardiomyopathy or low-risk surgery (i.e., breast surgery, dental surgery, endoscopic procedure, ophthalmic surgery, gynecological surgery, reconstructive surgery, minor orthopedic surgery, and minor urological surgery). The main outcomes were the in-hospital risk of MACCE: acute myocardial infarction (AMI) [ICD-10 code I21], heart failure (HF) [ICD-10 codes I50], ischemic stroke [ICD-10 code I63 or I64], and all-cause death.

Statistical analysis

Continuous variables were expressed as mean ± standard deviation and compared by the Mann–Whitney U test. Categorical variables were presented as absolute values and percentages and compared by the Chi-square test. There were missing values for predictors in this study, which were handled by multiple imputation with five imputations, and all predictor variables were included in the imputation model. After imputation was completed, the results of the analysis of the imputed data were merged according to Rubin's rules. Univariable logistic regression analyses were applied to identify the risk factors for MACCE during the perioperative period of high-risk noncardiac surgery in elderly patients. LASSO regression was used to screen the variables. Nomograms were constructed using the selected variables. The receiver operating characteristic (ROC) curve was plotted to evaluate the discriminatory ability of the nomograph model. The Hosmer–Lemeshow test was used to evaluate the goodness-of-fit of the models and the calibration curves were plotted. The net benefit was estimated by the decision curve analysis. Using the Areas under the receiver operating characteristic curves (AUC), the unclassified net reclassification improvement (NRI) and integrated discrimination improvement (IDI), we compared the performance of our model to the modified RCRI score. We employed five machine learning methods—Decision Tree, XGBoost, Support Vector Machine, K-nearest Neighbor, and Neural network(R package:rpart,rpart.plot,xgboost,e1071,sklearn,neuralnet)—for model establishment. A sample of 70% of the cohort generated randomly using a seed was applied for the training set; the remaining 30% was used for testing. The R software 4.0.2 was used for data analysis. P < 0.05 indicated statistical significance.

Results

MACCE and mortality

From January 1st, 2018, to June 30th, 2022, 40873 adult patients underwent surgery in our hospital. After screening, a total of 9739 patients were included. Among them, 84 cases had MACCE events. The incidence of MACCE and the mortality rate of patients undergoing noncardiac surgery increased with age (Table 1). In patients aged ≥ 85 years, the incidence of MACCE was 2.462% and the mortality rate was 0.821%.

Table 1 The incidence of MACCE and the mortality rate of patients at different ages

Patients without MACCE during hospitalization were firstly divided into five subgroups according to the operation type of the MACCE group: intraperitoneal surgery, thoracic surgery, peripheral arterioplasty, major orthopedic surgery and major urological surgery. Stratified randomization was used to protect against the possibility of imbalance with the MACCE group. Then, 285 cases stratified by the five subgroups were randomly selected as the control group. Finally, 342 patients (84 in the MACCE group and 258 in the control group) were included for further analysis.

3.2 Patient characteristics and verification of the modified RCRI scoreThe baseline characteristics of the MACCE and control groups are shown in Table 2. The mean age was significantly different between the two groups. The proportion of patients aged ≥ 85 years was significantly higher in the MACCE group than in the control group (P = 0.003). Compared to the controls, significantly more patients in the MACCE group had a history of coronary heart disease, ischemic chest pain, HF (especially HF with symptoms), arrhythmia, atrial fibrillation, valvular heart disease, nitrate therapy, and creatinine (Cr) levels of over 2.0 mg/dL (all P > 0.05). The blood transfusion and blood transfusion volume, intraoperative minimum diastolic pressure, minimum mean arterial pressure, intraoperative maximum heart rate, postoperative systolic pressure, and postoperative heart rate of the MACCE group were significantly different from those of the control group, while no significant difference was observed in the high-risk surgery type. Compared with the control group, patients with MACCE showed significantly larger left atrial anterior–posterior diameter, left atrial transverse diameter, left atrial long diameter, left ventricular mass, and left ventricular mass index, and lower left ventricular ejection fraction (LVEF).

Table 2 Baseline characteristics of recruited patients

Since the modified RCRI score did not take ischemic stroke and non-cardiac death into consideration, we validated the modified RCRI score after excluding 26 patients who experienced ischemic stroke or died due to non-cardiac cause during hospitalization. The area under the curve (AUC) of the modified RCRI score predicting the occurrence of MACCE in elderly patients undergoing noncardiac surgery in our study was 0.54 [95% confidence interval (0.419, 0.660)].

Risk factors

Univariable logistic regression analysis showed that the risk factors for perioperative MACCE in patients aged ≥ 65 years (Table 3) were as follows: age (especially ≥ 85 years), medical history (i.e., history of coronary heart disease, ischemic chest pain, nitrate therapy, HF, arrhythmia, heart valve disease, and Cr levels of over 2.0 mg/dL), blood transfusion, the amount of transfused blood, surgery conditions (i.e., high-risk surgery type, intraoperative minimum systolic blood pressure, intraoperative minimum diastolic blood pressure, intraoperative minimum mean arterial pressure, intraoperative maximum heart rate, postoperative systolic blood pressure, and postoperative heart rate), laboratory test results (i.e., ApoA-I, myocardial ischemia changes in electrocardiogram), left atrial anterior–posterior diameter, left atrial transverse diameter, left atrial long diameter, and LVEF).

Table 3 Univariablelogistic analysis of risk factors for perioperative MACCE

Contribution and performance of the new prediction nomogram

LASSO regression analysis revealed that age ≥ 85 years, history of ischemic chest pain, symptoms of decompensated heart failure, high-risk surgery, intraoperative minimum systolic blood pressure, postoperative systolic blood pressure, Cr levels over 2.0 mg/dL, left ventricular ejection fraction, and perioperative blood transfusion were risk factors for perioperative MACCE in elderly patients. An optimal nomograph model for predicting the risk of perioperative MACCE in elderly patients undergoing noncardiac surgery was constructed using the training set (70% of all recruited patients; 60 in the MACCE group and 181 in the control group) (Fig. 1a). The sum of the score of each variable was considered the total score of the patient, and a vertical line was drawn at the total score. The corresponding prediction probability was defined as the perioperative incidence of MACCE in patients with noncardiac surgery.

Fig. 1

The nomogram, ROC curve, calibration plot, and the decision curve of the new predictive model. a Images indicating the nomogram for each variable of the prediction model; b ROC curve of the new predictive model with the validation cohort; c Calibration plot for the new prediction model with the validation cohort; d Decision curve of the new predictive model. EF: Left ventricular ejection fraction;Cr 2.0 > mg/dL, preoperative serum creatinine > 2.0 mg/dL

The nomogram was internally validated by the test set population (30% of all recruited patients). The area under the curve (AUC) of the nomogram model predicting the occurrence of MACCE in elderly patients undergoing noncardiac surgery was 0.89 [95% confidence interval (0.818, 0.963)] (Fig. 1b). The calibration curve of the nomogram model was consistent with the actual curve, and it was confirmed by the Hosmer–Lemeshow goodness of fit test (χ2 = 4.9299, P = 0.765, Fig. 1c). The decision curve showed a large net benefit across the range of the MACCE risk of the new prediction nomogram (Fig. 1d).Compared with the modified RCRI score, the new prediction model had significant improvement in reclassification as assessed by the NRI (1.08 [95% CI, 0.835–1.334]) and IDI (0.307 [95% CI, 0.231–0.383]).

Five machine learning methods, including Decision Tree, XGBoost, Support Vector Machine, K-nearest Neighbor, and Neural network were used to establish prediction models using the training set, which were then validated on the test set. The five methods and the LASSO logistic regression results were compared with the ROC curve, calibration plot, decision curve, AUC,sensitivity, specificity, and F1 score of the test set (Fig. 2, Table 4). The XGBoost model was the best model when the same variables were selected (AUC = 0.903, Accuracy = 0.892).

Fig. 2

Comparison between the LASSO logistic regression and five machine learning methods. a AUC of the six methods. b Decision curve analysis evaluation of six methods. c Calibration plot of the six methods

Table 4 Performance evaluation and comparison of six models

In order to guide pre-operative cardiac evaluation and optimization of patients at elevated risk prior to surgery, we reconstructed the preoperative prediction model after removing the intraoperative and postoperative variables. Using LASSO logistic regression, we selected 5 strongest predictors: age ≥ 85 years, symptoms of decompensated HF, ischemic chest pain, high-risk type of surgery and Cr > 2.0 mg/dL. Integrating the 5 variables, we were able to build a nomogram for predicting in-hospital MACCEs using the training set (70% of all recruited patients) (Fig. 3a). The preoperative prediction nomogram was internally validated by the test set population (30% of all recruited patients). The ROC curve of the preoperative prediction nomogram is shown in Fig. 3b with an AUC of 0.720 [95% CI, 0.591–0.848] and the accuracy is 0.824 [95% CI, 0.735–0.892]. The sensitivity of preoperative prediction nomogram is 0.435 and the specificity is 0.937. The calibration plot of the preoperative model showed an overall good agreement between the predicted and observed risks, which was further supported by Hosmer–Lemeshow test (Fig. 3c, χ2 = 0. 857, p = 0.999). The decision curve showed a large net benefit across the range of the MACCE risk of the preoperative prediction nomogram (Fig. 3d).Compared with the modified RCRI score, the preoperative prediction model had improvement in reclassification as assessed by the NRI (0.639 [95% CI, 0.369–0.909]) and IDI (0.235[95% CI, 0.162–0.308]). But compared with the preoperative prediction model, our nomogram with intraoperative and postoperative variables were better in reclassification as assessed by the NRI (0.609 [95% CI, 0.330–0.888]) and IDI (0.071[95% CI, 0.028–0.115]).

Fig. 3

The nomogram, ROC curve, calibration plot, and the decision curve of the preoperative predictive model. a Images indicating the nomogram for each variable of the preoperative predictive model; b ROC curve of the preoperative predictive model with the validation cohort; c Calibration plot for the preoperative predictive model with the validation cohort; d Decision curve of the preoperative predictive model. Cr 2.0 > mg/dL, preoperative serum creatinine > 2.0 mg/dL

Discussion

Elderly patients undergoing noncardiac surgery have a higher chance of developing complications, such as surgical complications, and having worse general conditions compared to young and middle-aged patients [11, 12]. In this cohort, the incidence of perioperative MACCE and the mortality rate increased significantly with age. In patients aged ≥ 85 years, the incidence of perioperative MACCE and the mortality rate were as high as 2.463% and 0.821%, respectively. Therefore, more attention needs to be paid to the risk of perioperative MACCE in elderly patients undergoing high-risk noncardiac surgery.

The risk factors identified in this study for MACCE in elderly patients undergoing medium and high-risk noncardiac surgery are in line with previous reports. It is worth noting that blood transfusion and the amount of blood transfusion are independent predictors of perioperative MACCE [13]. Most patients who need blood transfusion and a large amount of blood transfusion have more pre-existing diseases (e.g., anemia), a larger surgical wound (e.g., after Whipple surgery), and more intraoperative and postoperative blood loss, all of which may increase the incidence of MACCE. Secondly, blood pressure plays a key role in maintaining the perfusion of all organs. Therefore, blood pressure management during the perioperative period is important. Previous studies have shown that hypotension is closely related to myocardial injury, myocardial infarction, renal injury, and death [14, 15]. The Perioperative Quality Initiative consensus statement on intraoperative blood pressure, risks, and outcomes of elective noncardiac surgery suggests that hypotension, regardless of its cause, even brief hypotension, is harmful [16]. In this study, we found that intraoperative minimum diastolic pressure, intraoperative minimum systolic pressure, intraoperative minimum mean arterial pressure, and postoperative systolic pressure of the MACCE group were significantly different from those of the controls. Thirdly, univariable logistic regression analysis demonstrated that high heart rate during surgery was a risk factor for MACCE, possibly due to increased myocardial oxygen consumption as the heart rate increases. These findings need to be validated in large-scale studies.

In this study, we further constructed and validated a model to predict the risk of perioperative MACCE in elderly patients undergoing medium and high-risk noncardiac surgery. The prediction model not only showed good prediction performance but also exhibited the following characteristics. Firstly, fewer predictive variables were included in this model, which improved its clinical applicability. The ACS NSQIP Surgical Risk Calculator contains 22 variables and requires a network calculator, while this prediction model only contains 9 variables and can be calculated by nomogram. Secondly, the included variables are easy to obtain in clinical settings, including age ≥ 85 years, medical history of ischemic chest pain and symptoms of decompensated HF, high-risk surgery, intraoperative minimum systolic blood pressure and postoperative systolic blood pressure, Cr levels over 2.0 mg/dL, LVEF, and perioperative blood transfusion. The levels of Cr can be measured in the routine renal function examination before surgery. LVEF can be detected by preoperative echocardiography without affecting patient’s condition, medical insurance cost, and the average length of stay. Thirdly, compared with RCRI and ACS NSQIP Surgical Risk Calculator, our model also included blood pressure and LVEF. LVEF is an independent influencing factor for perioperative adverse events and long-term mortality risk [17,18,19]. In contrast to medical history and clinical symptoms, LVEF is an objective variable. Lastly, the endpoints of this prediction model included ischemic stroke in addition to major adverse cardiovascular events. With the increase in the incidence rate of perioperative ischemic stroke in recent years, this model may better meet the clinical needs.

Machine learning is a subfield of artificial intelligence, which effectively learns the features of a large input data set and provides an alternative method for risk prediction. In this study, five algorithms based on machine learning, including Decision Tree, XGBoost, Support Vector Machine, K-nearest Neighbor, and Neural Network, were used to evaluate the prediction model for the perioperative risk of MACCE. The prediction ability of the XGBoost model was slightly better than that of the Lasso logistic regression model (AUC = 0.903) when the same variables were selected.

This study also established a preoperative prediction model to facilitate clinicians' preoperative risk assessment. Combining the two models, preoperative and postoperative, could lead to a better risk assessment and postoperative management. Both models compared with the modified RCRI score had improvement in reclassification.

This study has some limitations. First, it was a single-center study, and the results were not externally validated. Secondly, the sample size was small due to the low prevalence of MACCE. Even if the random sampling method was adopted, selection bias might occur. Further studies with a larger sample size are needed. Thirdly, although compared with the preoperative prediction model, the prediction model with intraoperative and postoperative variables was better in reclassification as assessed by the NRI and IDI, but the utility of it is limited. Finally, this was a retrospective study. Further studies with a prospective design are needed to explore the potential risk factors for perioperative MACCE and to establish a model with better prediction performance.

Conclusion

This study identified the risk factors for perioperative MACCE in elderly patients undergoing medium and high-risk noncardiac surgery, which provided a basis for better management and prevention of MACCE. A new model for predicting the risk of perioperative MACCE was constructed and validated. It showed good prediction performance and may be used as a decision-making assistant tool for clinicians. These data lay a foundation for future large-scale prospective clinical trials.