Abstract

Methods: In the study, classification and the risk scoring were performed with the four different systems for 1,950 patients (1,038 males, 912 females; mean age 5.5 months; range, 1 day to 18 years) who were administered congenital heart surgery between 1 October 2012 and 31 December 2016. The hospital mortality and morbidity were calculated for each category from the four models. The discriminatory ability of the models was determined by calculating the area under the receiver operating characteristic curve and the receiver operating characteristic curves of the four models were compared.

Results: Median weight of the patients was 7.2 kg (range, 1.8-80 kg). Among the patients, 53% were males and 47.5% were younger than one year of age. Of totally 1,950 operations, mortality was observed in 149 (7.6%) and morbidity was observed in 541 (27.7%). Areas under the receiver operating characteristic curve for mortality were 0.803, 0.795, 0.729, and 0.712 for the Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories, Aristotle Comprehensive Complexity, Risk Adjustment in Congenital Heart Surgery, and Aristotle Basic Complexity scores, respectively. Areas under the receiver operating characteristic curve for morbidity were 0.732, 0.731, 0.730, and 0.685 for the Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories, Risk Adjustment in Congenital Heart Surgery, Aristotle Comprehensive Complexity, and Aristotle Basic Complexity scores, respectively.

Conclusion: Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories, Risk Adjustment in Congenital Heart Surgery, Aristotle Basic Complexity, and Aristotle Comprehensive Complexity score systems were effective in predicting the morbidities and mortalities of patients who underwent congenital heart surgery and evaluating the performance of the surgical centers. Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories were on the forefront due to high feasibility and performance. Aristotle Basic Complexity score system had the lowest performance. Combinations of systems will provide the most benefit during evaluation of results.

Congenital heart diseases have various types and numbers of pathologies. Patients undergo either corrective or palliative surgeries depending on the problem. In addition, many underlying factors such as low birth weight, genetic syndromes, or preoperative clinical conditions affect the success of surgery.

It is difficult to establish a risk stratification system of nomenclature that is universally accepted for congenital heart diseases.[2] There are different risk scoring systems for pediatric cardiac surgery to determine the risks and difficulty levels. In addition, healthcare centers may use scoring systems to predict their mortality and morbidity rates; thus, determine their current situation compared to national and international healthcare centers and prepare for necessary regulations.[3-6] Risk Adjustment in Congenital Heart Surgery (RACHS-1), Aristotle Basic Complexity (ABC) score, Aristotle Comprehensive Complexity (ACC) score, and Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories (STS-EACTS MC) systems are most commonly used for this purpose.[3-5] Although these systems are strongly related to hospital mortality and morbidity rates, studies show that they are different in terms of effectiveness.[5-8] Furthermore, studies that compare all four scoring systems are limited. Therefore, in this study, we aimed to evaluate the surgical results of our clinic according to presumption systems of RACHS-1, ABC score, ACC score, and STS-EACTS MC and to compare the efficiency of these systems in predicting morbidity and mortality.

Methods

The inclusion criterion was to have undergone operation due to a congenital or acquired heart disease (either corrective or palliative) under the age of 18. Cases that could not be categorized in any of the four scoring systems, such as permanent pacemaker implantations, and those that were transferred to another unit for any reason were excluded. Data from patient records regarding age, gender, weight, year of surgery, diagnosis, presence of syndromes, type of surgery, and use of cardiopulmonary bypass (CPB) were obtained. Cases were separated according to their age groups as newborn (1-28 days), infant (29 days-1 year), toddler (1-12 years) and adolescent (12-18 years).

A team consisting 11 pediatric cardiologists and pediatric cardiac surgeons from Boston Children’s Hospital created the RACHS-1 model, which contains 207 procedures. The RACHS-1 system scale ranges from one to six. Three additional clinical factors (age, prematurity and non-cardiac congenital structural abnormalities) complement the model.[3]

In 1999, 50 cardiac surgeons led by Lacour-Gayet from 23 different countries developed the ABC score, which is based on three factors: potential for mortality, potential for morbidity and anticipated technical difficulty. For the ABC system, the scale ranges from 1.5 to 15. The ACC further adjusts complexity according to specific patient characteristics. It includes two categories of complexity factors as procedure dependent and procedure independent factors and a corresponding basic complexity level between one and four (level 1 [1.5 to 5.9]; level 2 [6.0 to 7.9]; level 3 [8.0 to 9.9] and level 4 [10.0 to 15.0]).[4]

Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories are the latest system found in 2008. The mortality risk was estimated for 148 procedure types, using real data from 77,294 patients (33,360 patients from the EACTS and 43,934 patients from the STS) between 2002 and 2007. Using Bayesian statistics that fit the data for small denominators, mortality rates were calculated for each procedure. For the STSEACTS MC, the scale range from 0.1 to 5.0, and a corresponding mortality category level between 1 and 5 is assigned (level 1 [0.1 to 0.3]; level 2 [0.4 to 0.7]; level 3 [0.8 to 1.2]; level 4 [1.3 to 2.6] and level 5 [2.9 to 5.0]).[5]

In each scoring system, a higher score indicates a higher risk of mortality. For patients undergoing multiple procedures, the procedure with the highest level was scored. Every single case’s risk scores were calculated using RACHS-1, ABC, ACC, and STSEACTS MC scoring systems.

In this study, our two primary outcomes were operative mortality and hospital morbidity. Operative mortality included all the deaths occurred during the hospital stay when the operation was performed and the deaths occurred after discharge within 30 days of the procedure. Intensive care unit stay longer than seven days was identified as hospital morbidity.

Statistical analysis
Data were analyzed using SPSS for Windows version 15.0 software (SPSS Inc., Chicago, IL, USA). Descriptive statistics were performed in relation to the distribution of variables and characteristics of the study population. Categorical variables were represented as frequencies; the numeric variables as mean or median with the respective measures of dispersion. Analysis of the discriminatory ability of the surgical risk stratification methods were performed using the C statistic comparison with receiver operating characteristic (ROC) curves of the four methods.[7]

Results

Table 1: Demographic characteristics of patients undergoing cardiovascular surgery

Mortality and morbidity rates were 7.6% (n=149) and 27.7% (n=541), respectively. Mortality by years was 9.5% in 2012, 8.9% in 2013, 8.7% in 2014, 6% in 2015 and 5.9% in 2016. Seventeen cases could not be categorized in RACHS-1. In all scoring systems, as the category or the level increased, the mortality and the morbidity increased accordingly. The mortality and the morbidity rates were shown in Table 2.

Table 2: Distribution of scores over cohort

The results of the area under the ROC curve for mortality and morbidity are shown in Figure 1a, b. The RACHS-1 categories, ABC, ACC and STS-EACTS MC had satisfactory performance (above 0.650).

Figure 1: Receiver operating characteristic curves: (a) Mortality receiver operating characteristic curves for comparison. (b) Morbidity receiver operating characteristic curves for comparison.
ROC: Receiver operating characteristic; RACHS-1: Risk Adjustment in Congenital Heart Surgery; STS-EACTS MC: Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories; ACC: Aristotle Comprehensive Complexity score; ABC: Aristotle Basic Complexity score.

There was no statistical difference between the four forms of categorization and the areas under the ROC curve of the four methods for the discriminatory capacity for hospital mortality (Table 3).

Table 3: Results regarding area under receiver operating characteristic curve between methods concerning mortality and morbidity outcomes

Discussion

There are different types of surgeries used in various complex congenital heart diseases. Therefore, scoring systems are needed to evaluate mortality and morbidity. It is difficult to establish a universally accepted risk stratification system of nomenclature as congenital heart surgery involves approximately 200 different diagnostic topics and procedures. Categorization of these into groups or relatively homogeneous strata is necessary to compare the outcomes within each category. In pediatric cardiac surgery, mortality analysis without stratification of complexity is a failure. This risk stratification has been identified essential for assessing and improving the quality.[15,16]

Risk Adjustment in Congenital Heart Surgery has been the first system used for this purpose. It is rather an easier model, because it needs less data. Although it does not contain all cardiac procedures and is based on personal foresight, it is widely used to determine mortality. RACHS-1 was first described by Jenkins et al.[3] They divided cardiac diseases into six categories. Expected mortalities in category 1, 2, 3, 4 and 6 are 0.4%, 3.8%, 8.5%, 19.4%, and 47.7%, respectively. For category 5, there is no estimation of mortality.

Cavalcante et al.[14] evaluated 3,071 cases and showed that mortality ratios according to RACHS-1 was 1.8% in category 1, 5.5% in category 2, 14.9% in category 3, 32.5% in category 4, and 68.8% in category 6. In our study, 1,931 of 1,950 cases were suitable for RACHS-1. Majority of the cases were in category 2 (n=838). Mortality rates were as follows: 0.5% in category 1, 3.4% in category 2, 10.5% in category 3, 13.8% in category 4, and 43.1% in category 6.

European Association for Cardiothoracic Surgery defined Aristotle scoring system as two separate scoring systems, namely, ABC and ACC, in 2014.[4] The first score was the Basic score, which adjusted only for the complexity of the procedure. The second score was the Comprehensive score, which took into account specific procedure-dependent and procedureindependent patient characteristics. It is divided into four levels according to difficulty. Lacour-Gayet et al.[4] indicated mortality and morbidity in ABC and ACC as follows; <1% in level 1, 1-5% in level 2, 5-10% in level 3, and 10-20% in level 4. In our study, mortality for ABC and ACC was 0.7%-0.8% for level 1, 5.4%-3.7% for level 2, 7.9%-5% for level 3, and 19.7-15.8% for level 4, respectively. Our results were similar to predicted values.

Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories is the most recent scoring system estimating mortality based on processing data from a database containing 148 different procedures. It divides into five subgroups according to difficulty level. Mortality rates are as follows; 0.8% in level 1, 2.6% in level 2, 5% in level 3, 9.9% in level 4, and 23.1% in level 5. In our study, mortality rates were 0.6% in level 1, 2% in level 2, 8.7% in level 3, 16% in level 4, and 55% in level 5. An evaluation of all the scoring systems showed that expected rates were obtained according to the results of RACHS-1, ABC, and ACC. Based on STS-EACTS MC, we may suggest that regulations should be developed to improve the results in levels 3-5.

Different results were achieved when the scoring systems were compared in different studies. Al-Radi et al.[15] evaluated t he m ortality p rediction of R ACHS-1 and ABC in 11,438 operated congenital heart disease patients. They both indicated strong predictive effect while the best scoring system was RACHS-1 (ROC curve 0.733&0.698).

Furthermore, Joshi et al.[8] retrospectively evaluated the mortality prediction using ROC analysis in 1,150 cases. The results were 0.677 (95% CI: 0.61-0.73) for ABC, 0.704 (95% CI: 0.64-0.76) for ACC, and 0.607 (95% CI: 0.55-0.66) for RACHS-1. They claimed that ACC had better results than both ABC and RACHS-1.

Moreover, Bojan et al.[12] compared ACC and RACHS-1 on 1,384 cases and suggested that ACC had superior results. They also stated that when a corrected model for age, prematurity and extracardiac anomalies was used, then RACHS-1 had an equal efficacy.

Cavalcanti et al.[16] compared mortality in three systems (STS-EACTS MC, ACC, and RACHS-1) on 360 cases who were under 18 years of age in Brazil. The three models had similar accuracy by calculating the area under the ROC curve: RACHS-1: 0.738, STS-EACTS MC: 0.739, and ACC: 0.766.

In addition, O’Brien[5] compared mortality prediction of STS-EACTS MC to ACC and RACHS-1. The discriminatory capacity of STS-EACTS MC by area under the ROC curve (0.778) was higher than RACHS-1 (0.745) and ABC (0.687).

The association of mortality outcomes with annual hospital volume has been substantiated in the literature. Centers with <150 cases/year had an odds ratio of 1.59 for having higher operative mortality. Certainly, this reflects the learning curve for surgeons, perioperative cardiologists, and anesthesiologists.[17,18] Our study supports these opinions, with mortality being 9.5% in 2012 and decreasing to 5.9% in 2016.

Some studies have emphasized that scoring systems can be used as a predictor of morbidity. Larsen et al.[19] applied the RACHS-1 score to rank 957 procedures in Denmark and concluded that it is a good predictor of morbidity.

On the other hand, Kogon and Oster[11] evaluated 458 cases of congenital heart disease patients who were over 18 years old. For prolonged length of stay, areas under the ROC curve were 0.82, 0.76, and 0.61 for the Aristotle, STS-EACTS MC, and RACHS-1 scores, respectively. In our study, the morbidity predictability of the four scoring systems was high, with STS-EACTS MC having the best predictive score (0.732). This was followed by RACHS-1 (0.731) and ACC (0.730). The efficacy of ABC was lower than the others (0.685).

There were some limitations to our study. Primarily, it was a retrospective and single-centered study. Furthermore, the number of operations performed in the first year was low compared to the other years. In addition, there were three different surgical teams performing the surgeries.

In conclusion, all four scoring systems that assist in resolving the complexity of congenital heart disease and difficulty of surgical treatment were successful in determining hospital mortality and morbidity. However, although an evaluation of the comparative results showed that the scoring systems provided similar results, the Society of Thoracic Surgeons and European Association for Cardiothoracic Surgery mortality categories scoring system was superior than the others.

Declaration of conflicting interests
The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

Funding
The authors received no financial support for the research and/or authorship of this article.

References

1) van der Linde D, Konings EE, Slager MA, Witsenburg M, Helbing WA, Takkenberg JJ, et al. Birth prevalence of congenital heart disease worldwide: a systematic review and meta-analysis. J Am Coll Cardiol 2011;58:2241-7.

2) Jacobs JP, Jacobs ML, Mavroudis C, Backer CL, Lacour- Gayet FG, Tchervenkov CI, et al. Nomenclature and databases for the surgical treatment of congenital cardiac disease--an updated primer and an analysis of opportunities for improvement. Cardiol Young 2008;18 Suppl 2:38-62.

3) Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg 2002;123:110-8.

4) Lacour-Gayet F, Clarke D, Jacobs J, Comas J, Daebritz S, Daenen W, et al. The Aristotle score: a complexity-adjusted method to evaluate surgical results. Eur J Cardiothorac Surg 2004;25:911-24.

5) O’Brien SM, Clarke DR, Jacobs JP, Jacobs ML, Lacour- Gayet FG, Pizarro C, et al. An empirically based tool for analyzing mortality associated with congenital heart surgery. J Thorac Cardiovasc Surg 2009;138:1139-53.

6) Erek E, Yılmaz B, Kaya M, Onan İS, Şen O, Öz K, et al. Analysis of results according to the Aristotle scoring system in congenital heart surgery. Turk Gogus Kalp Dama 2014;22:509-16.

7) Sabuncu T, Demircin M, Doğan R, Yılmaz M, Aypar Ü, Paşaoğlu İ. Comparison of risk scoring systems in congenital heart surgery. Turk J Pediatr 2016;58:512-7.

8) Joshi SS, Anthony G, Manasa D, Ashwini T, Jagadeesh AM, Borde DP, et al. Predicting mortality after congenital heart surgeries: evaluation of the Aristotle and Risk Adjustement in Congenital Heart Surgery-1 risk prediction scoring systems: a retrospective single center analysis of 1150 patients. Ann Card Anaesth 2014;17:266-70.

9) DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837-45.

10) Jacobs JP, Jacobs ML, Lacour-Gayet FG, Jenkins KJ, Gauvreau K, Bacha E, et al. Stratification of complexity improves the utility and accuracy of outcomes analysis in a Multi-Institutional Congenital Heart Surgery Database: Application of the Risk Adjustment in Congenital Heart Surgery (RACHS-1) and Aristotle Systems in the Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database. Pediatr Cardiol 2009;30:1117-30.

11) Kogon B, Oster M. Assessing surgical risk for adults with congenital heart disease: are pediatric scoring systems appropriate? J Thorac Cardiovasc Surg 2014;147:666-71.

12) Bojan M, Gerelli S, Gioanni S, Pouard P, Vouhé P. Comparative study of the Aristotle Comprehensive Complexity and the Risk Adjustment in Congenital Heart Surgery scores. Ann Thorac Surg 2011;92:949-56.

13) Heinrichs J, Sinzobahamvya N, Arenz C, Kallikourdis A, Photiadis J, Schindler E, et al. Surgical management of congenital heart disease: evaluation according to the Aristotle score. Eur J Cardiothorac Surg 2010;37:210-7.

14) Cavalcante CT, de Souza NMG, Pinto VC Júnior, Branco KM, Pompeu RG, Teles AC, et al. Analysis of Surgical Mortality for Congenital Heart Defects Using RACHS-1 Risk Score in a Brazilian Single Center. Braz J Cardiovasc Surg 2016;31:219-225.

15) Al-Radi OO, Harrell FE Jr, Caldarone CA, McCrindle BW, Jacobs JP, Williams MG, et al. Case complexity scores in congenital heart surgery: a comparative study of the Aristotle Basic Complexity score and the Risk Adjustment in Congenital Heart Surgery (RACHS-1) system. J Thorac Cardiovasc Surg 2007;133:865-75.

16) Cavalcanti PE, Sá MP, Santos CA, Esmeraldo IM, Chaves ML, Lins RF, et al. Stratification of complexity in congenital heart surgery: comparative study of the Risk Adjustment for Congenital Heart Surgery (RACHS-1) method, Aristotle basic score and Society of Thoracic Surgeons-European Association for Cardio- Thoracic Surgery (STS-EACTS) mortality score. Rev Bras Cir Cardiovasc 2015;30:148-58.

17) Spiegelhalter DJ. Mortality and volume of cases in paediatric cardiac surgery: retrospective study based on routinely collected data. BMJ 2002;324:261-3.

18) Pasquali SK, Li JS, Burstein DS, Sheng S, O’Brien SM, Jacobs ML, et al. Association of center volume with mortality and complications in pediatric heart surgery. Pediatrics 2012;129:e370-6.

19) Larsen SH, Pedersen J, Jacobsen J, Johnsen SP, Hansen OK, Hjortdal V. The RACHS-1 risk categories reflect mortality and length of stay in a Danish population of children operated for congenital heart disease. Eur J Cardiothorac Surg 2005;28:877-81.