Log in | Register

Comparison between SAPS II and SAPS 3 in predicting hospital mortality in a cohort of 103 Italian ICUs. Is new always better?

Daniele Poole| Carlotta Rossi| Nicola Latronico| Giancarlo Rossi| Stefano Finazzi| Guido Bertolini
Original
Volume 38, Issue 8 / August , 2012

Pages 1280 - 1288

Abstract

Purpose

More recent severity scores should be more reliable than older ones because they account for the improvement in medical care over time. To provide more insight into this issue, we compared the predictive ability of the Simplified Acute Physiology Score (SAPS) II and SAPS 3 (originally developed from data collected in 1991–1992 and 2002, respectively) on a sample of critically ill patients.

Methods

This was a prospective observational study on 3,661 patients from 103 Italian intensive care units. Standardized mortality ratios (SMRs) were calculated. Assessment of calibration across risk classes was performed using the GiViTI calibration belt. Discrimination was evaluated by means of the area under the receiver operating characteristic analysis.

Results

Both scores were shown to discriminate fairly. SAPS 3 largely overpredicted mortality, more than SAPS II (SMR 0.63, 95 % CI 0.60–0.66 vs. 0.87, 95 % CI 0.83–0.91). This result was consistent and statistically significant across all risk classes for SAPS 3. SAPS II did not show relevant deviations from ideal calibration in the first two deciles of risk, whereas in higher-risk classes it overpredicted mortality.

Conclusions

Both scores provided unreliable predictions, but unexpectedly the newer SAPS 3 turned out to overpredict mortality more than the older SAPS II.

Keywords

References

  1. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A et al (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1636
    • View reference on PubMed
    • View reference on publisher's website
  2. Zimmerman JE, Kramer AA, McNair DS, Malila FM (2006) Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med 34:1297–1310
    • View reference on PubMed
    • View reference on publisher's website
  3. Sirio CA, Shepardson LB, Rotondi AJ, Cooper GS, Angus DC, Harper DL, Rosenthal GE (1999) Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure: implications for critical care delivery from Cleveland health quality choice. Chest 115:793–801
    • View reference on PubMed
    • View reference on publisher's website
  4. Minne L, Eslami S, de Keizer N, de Jonge E, de Rooij SE, Abu-Hanna A (2012) Effect of changes over time in the performance of a customized SAPS-II model on the quality of care assessment. Intensive Care Med 38:40–46
  5. Le Gall JR, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270:2957–2963
    • View reference on PubMed
    • View reference on publisher's website
  6. Metnitz PG, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR (2005) SAPS 3–From evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohort description. Intensive Care Med 31:1336–1344
  7. Capuzzo M, Moreno RP, Le Gall JR (2008) Outcome prediction in critical care: the simplified acute physiology score models. Curr Opin Crit Care 14:485–490
    • View reference on PubMed
    • View reference on publisher's website
  8. Boffelli S, Rossi C, Anghileri A, Giardino M, Carnevale L, Messina M, Neri M, Langer M, Bertolini G; Italian Group for the Evaluation of Interventions in Intensive Care Medicine (2006) Continuous quality improvement in intensive care medicine. The GiViTI Margherita Project—Report 2005. Minerva Anestesiol 72:419–432
  9. Poole D, Rossi C, Anghileri A, Giardino M, Latronico N, Radrizzani D, Langer M, Bertolini G (2009) External validation of the Simplified Acute Physiology Score (SAPS) 3 in a cohort of 28,357 patients from 147 Italian intensive care units. Intensive Care Med 35:1916–1924
  10. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR (2005) SAPS 3–From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 31:1345–1355
  11. Campbell G (1994) Advances in statistical methodology for the evaluation of diagnostic and laboratory tests. Stat Med 13:499–508
    • View reference on PubMed
    • View reference on publisher's website
  12. Lemeshow S, Hosmer DW Jr (1982) A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 115:92–106
    • View reference on PubMed
  13. Bertolini G, D’Amico R, Nardi D, Tinazzi A, Apolone G (2000) One model, several results: the paradox of the Hosmer–Lemeshow goodness-of-fit test for the logistic regression model. J Epidemiol Biostat 5:251–253
    • View reference on PubMed
  14. Kramer AA, Zimmerman JE (2007) Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit Care Med 35:2052–2056
    • View reference on PubMed
    • View reference on publisher's website
  15. Cox D (1958) Two further applications of a model for a method of binary regression. Biometrika 45:562–565
  16. Finazzi S, Poole D, Luciani D, Cogo PE, Bertolini G (2011) Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLoS ONE 6:e16110
    • View reference on PubMed
    • View reference on publisher's website
  17. Moreno R, Apolone G, Miranda DR (1998) Evaluation of the uniformity of fit of general outcome prediction models. Intensive Care Med 24:40–47
  18. Strand K, Soreide E, Aardal S, Flaatten H (2009) A comparison of SAPS II and SAPS 3 in a Norwegian intensive care unit population. Acta Anaesthesiol Scand 53:595–600
    • View reference on PubMed
    • View reference on publisher's website
  19. Capuzzo M, Scaramuzza A, Vaccarini B, Gilli G, Zannoli S, Farabegoli L, Felisatti G, Davanzo E, Alvisi R (2009) Validation of SAPS 3 Admission Score and comparison with SAPS II. Acta Anaesthesiol Scand 53:589–594
    • View reference on PubMed
    • View reference on publisher's website
  20. Soares M, Silva UV, Teles JM, Silva E, Caruso P, Lobo SM, Dal Pizzol F, Azevedo LP, de Carvalho FB, Salluh JI (2010) Validation of four prognostic scores in patients with cancer admitted to Brazilian intensive care units: results from a prospective multicenter study. Intensive Care Med 36:1188–1195
  21. Ledoux D, Canivet JL, Preiser JC, Lefrancq J, Damas P (2008) SAPS 3 admission score: an external validation in a general intensive care population. Intensive Care Med 34:1873–1877
  22. Murphy-Filkins R, Teres D, Lemeshow S, Hosmer DW (1996) Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. Crit Care Med 24:1968–1973
    • View reference on PubMed
    • View reference on publisher's website
  23. Miller ME, Hui SL, Tierney WM (1991) Validation techniques for logistic regression models. Stat Med 10:1213–1226
    • View reference on PubMed
    • View reference on publisher's website
  24. Metnitz PG, Lang T, Vesely H, Valentin A, Le Gall JR (2000) Ratios of observed to expected mortality are affected by differences in case mix and quality of care. Intensive Care Med 26:1466–1472
  25. Nimgaonkar A, Karnad DR, Sudarshan S, Ohno-Machado L, Kohane I (2004) Prediction of mortality in an Indian intensive care unit. Comparison between APACHE II and artificial neural networks. Intensive Care Med 30:248–253
  26. Zimmerman JE, Wagner DP, Draper EA, Wright L, Alzola C, Knaus WA (1998) Evaluation of acute physiology and chronic health evaluation III predictions of hospital mortality in an independent database. Crit Care Med 26:1317–1326
    • View reference on PubMed
    • View reference on publisher's website
  27. Babyak MA (2004) What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med 66:411–421
    • View reference on PubMed
    • View reference on publisher's website

Sign In

Connect with ICM

Top 5 Articles Editors Picks Supplement