Log in | Register

Prediction of long-term mortality in ICU patients: model validation and assessing the effect of using in-hospital versus long-term mortality on benchmarking

Sylvia Brinkman| Ameen Abu-Hanna| Evert de Jonge| Nicolette F. de Keizer
Original
Volume 39, Issue 11 / November , 2013

Pages 1925 - 1931

Abstract

Purpose

To analyze the influence of using mortality 1, 3, and 6 months after intensive care unit (ICU) admission instead of in-hospital mortality on the quality indicator standardized mortality ratio (SMR).

Methods

A cohort study of 77,616 patients admitted to 44 Dutch mixed ICUs between 1 January 2008 and 1 July 2011. Four Acute Physiology and Chronic Health Evaluation (APACHE) IV models were customized to predict in-hospital mortality and mortality 1, 3, and 6 months after ICU admission. Models’ performance, the SMR and associated SMR rank position of the ICUs were assessed by bootstrapping.

Results

The customized APACHE IV models can be used for prediction of in-hospital mortality as well as for mortality 1, 3, and 6 months after ICU admission. When SMR based on mortality 1, 3 or 6 months after ICU admission was used instead of in-hospital SMR, 23, 36, and 30 % of the ICUs, respectively, received a significantly different SMR. The percentages of patients discharged from ICU to another medical facility outside the hospital or to home had a significant influence on the difference in SMR rank position if mortality 1 month after ICU admission was used instead of in-hospital mortality.

Conclusions

The SMR and SMR rank position of ICUs were significantly influenced by the chosen endpoint of follow-up. Case-mix-adjusted in-hospital mortality is still influenced by discharge policies, therefore SMR based on mortality at a fixed time point after ICU admission should preferably be used as a quality indicator for benchmarking purposes.

Keywords

References

  1. de Vos M, Graafmans W, Keesman E, Westert G et al (2007) Quality measurement at intensive care units: which indicators should we use? J Crit Care 22:267–274
    • View reference on PubMed
    • View reference on publisher's website
  2. Sirio CA, Shepardson LB, Rotondi AJ, Cooper GS et al (1999) Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure: implications for critical care delivery from Cleveland health quality choice. Chest 115:793–801
    • View reference on PubMed
    • View reference on publisher's website
  3. Zimmerman JE, Kramer AA, McNair DS, Malila FM (2006) Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med 34:1297–1310
    • View reference on PubMed
    • View reference on publisher's website
  4. Brinkman S, de Jonge E, Abu-Hanna A, De Lange DW et al (2012) The use of linked registries to assess long-term mortality of ICU patients. Stud Health Technol Inform 180:230–234
    • View reference on PubMed
  5. Dutch National Intensive Care Evaluation (NICE) foundation (2013). http://www.stichting-nice.nl. Accessed 1 June 2013
  6. Vektis (2013). http://www.vektis.nl. Accessed 1 Jan 2013
  7. Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, de Jonge E et al (2013) Determinants of mortality after hospital discharge in ICU patients: literature review and Dutch cohort study. Crit Care Med 41:1237–1251
    • View reference on PubMed
    • View reference on publisher's website
  8. Harrell FE Jr (2001) Regression modeling strategies, with applications to linear models, logistic regression, and survival analysis. Springer, New York
  9. Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36
    • View reference on PubMed
  10. Hilden J, Habbema JD, Bjerregaard B (1978) The measurement of performance in probabilistic diagnosis. III. Methods based on continuous functions of the diagnostic probabilities. Methods Inf Med 17:238–246
    • View reference on PubMed
  11. Salter K, Jutai JW, Teasell R, Foley NC et al (2005) Issues for selection of outcome measures in stroke rehabilitation: ICF participation. Disabil Rehabil 27:507–528
    • View reference on PubMed
    • View reference on publisher's website
  12. Biancari F, Vasques F, Mikkola R, Martin M et al (2012) Validation of EuroSCORE II in patients undergoing coronary artery bypass surgery. Ann Thorac Surg 93:1930–1935
    • View reference on PubMed
    • View reference on publisher's website
  13. Efron B (1983) Estimating the error rate of a prediction rule: improvement on cross-validation. Am Stat Assoc 78:316–331
    • View reference on publisher's website
  14. Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S (1997) A comparison of goodness-of-fit tests for the logistic regression model. Stat Med 16:965–980
    • View reference on PubMed
    • View reference on publisher's website
  15. Zhu BP, Lemeshow S, Hosmer DW, Klar J et al (1996) Factors affecting the performance of the models in the mortality probability model II system and strategies of customization: a simulation study. Crit Care Med 24:57–63
    • View reference on PubMed
    • View reference on publisher's website
  16. Knaus WA, Harrell FE Jr, Lynn J, Goldman L et al (1995) The SUPPORT prognostic model. Objective estimates of survival for seriously ill hospitalized adults. Study to understand prognoses and preferences for outcomes and risks of treatments. Ann Intern Med 122:191–203
    • View reference on PubMed
    • View reference on publisher's website
  17. Ho KM, Knuiman M, Finn J, Webb SA (2008) Estimating long-term survival of critically ill patients: the PREDICT model. PLoS ONE 3:e3226
    • View reference on PubMed
    • View reference on publisher's website
  18. Brinkman S, Abu-Hanna A, van der Veen A, de Jonge E et al (2012) A comparison of the performance of a model based on administrative data and a model based on clinical data: effect of severity of illness on standardized mortality ratios of intensive care units. Crit Care Med 40:373–378
    • View reference on PubMed
    • View reference on publisher's website
  19. Bakhshi-Raiez F, Peek N, Bosman RJ, de Jonge E et al (2007) The impact of different prognostic models and their customization on institutional comparison of intensive care units. Crit Care Med 35:2553–2560
    • View reference on PubMed
    • View reference on publisher's website
  20. Lilford R, Pronovost P (2010) Using hospital mortality rates to judge hospital performance: a bad idea that just won’t go away. BMJ 340:c2016
    • View reference on PubMed
    • View reference on publisher's website
  21. Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, de Jonge E et al (2011) External validation of acute physiology and chronic health evaluation IV in Dutch intensive care units and comparison with acute physiology and chronic health evaluation II and simplified acute physiology score II. J Crit Care 26:105–108
    • View reference on PubMed
    • View reference on publisher's website
  22. Tromp M, Ravelli AC, Bonsel GJ, Hasman A et al (2011) Results from simulated data sets: probabilistic record linkage outperforms deterministic record linkage. J Clin Epidemiol 64:565–572
    • View reference on PubMed
    • View reference on publisher's website

Sign In

Connect with ICM

Top 5 Articles Editors Picks Supplement