Log in | Register

Mortality prediction models for pediatric intensive care: comparison of overall and subgroup specific performance

Idse H. E. Visser| Jan A. Hazelzet| Marcel J. I. J. Albers| Carin W. M. Verlaat| Karin Hogenbirk| Job B. van Woensel| Marc van Heerde| Dick A. van Waardenburg| Nicolaas J. G. Jansen| Ewout W. Steyerberg
Pediatric Original
Volume 39, Issue 5 / May , 2013

Pages 942 - 950

Abstract

Aim

To validate paediatric index of mortality (PIM) and pediatric risk of mortality (PRISM) models within the overall population as well as in specific subgroups in pediatric intensive care units (PICUs).

Methods

Variants of PIM and PRISM prediction models were compared with respect to calibration (agreement between predicted risks and observed mortality) and discrimination (area under the receiver operating characteristic curve, AUC). We considered performance in the overall study population and in subgroups, defined by diagnoses, age and urgency at admission, and length of stay (LoS) at the PICU. We analyzed data from consecutive patients younger than 16 years admitted to the eight PICUs in the Netherlands between February 2006 and October 2009. Patients referred to another ICU or deceased within 2 h after admission were excluded.

Results

A total of 12,040 admissions were included, with 412 deaths. Variants of PIM2 were best calibrated. All models discriminated well, also in patients <28 days of age (neonates), with overall higher AUC for PRISM variants (PIM = 0.83, PIM2 = 0.85, PIM2-ANZ06 = 0.86, PIM2-ANZ08 = 0.85, PRISM = 0.88, PRISM3-24 = 0.90). Best discrimination for PRISM3-24 was confirmed in 13 out of 14 subgroup categories. After recalibration PRISM3-24 predicted accurately in most (12 out of 14) categories. Discrimination was poorer for all models (AUC < 0.73) after LoS of >6 days at the PICU.

Conclusion

All models discriminated well, also in most subgroups including neonates, but had difficulties predicting mortality for patients >6 days at the PICU. In a western European setting both the PIM2(-ANZ06) or a recalibrated version of PRISM3-24 are suited for overall individualized risk prediction.

Keywords

References

  1. Yeh TS, Pollack MM, Ruttimann UE, Holbrook PR, Fields AI (1984) Validation of a physiologic stability index for use in critically ill infants and children. Pediatr Res 18(5):445–451
    • View reference on PubMed
    • View reference on publisher's website
  2. Pollack MM, Ruttimann UE, Getson PR (1988) Pediatric risk of mortality (PRISM) score. Crit Care Med 16(11):1110–1116
    • View reference on PubMed
    • View reference on publisher's website
  3. Examples of several countries with PICU registries, Australia and New-Zealand: ANZPIC Registry http://www.anzics.com.au/anzpicr. Accessed 8 Feb 2013; Netherlands: Dutch PICE registry http://www.pice.nl; Portugal: REUNIR-CIP http://newdbserver.med.up.pt/web.care/reunir/index.html; Italy: TIPNet http://tipnet.cineca.it; United Kingdom: PICANet http://www.picanet.org.uk; United States of America: the Virtual PICU Systems LLC (VPS) https://portal.myvps.org/
  4. Visser DY, Jansen NJ, Ijland MM, de Koning TJ, van Hasselt PM (2011) Intracranial bleeding due to vitamin K deficiency: advantages of using a pediatric intensive care registry. Intensive Care Med 37(6):1014–1020. doi:10.1007/s00134-011-2175-7
  5. Peeters B, Jansen NJ, Bollen CW, van Vught AJ, van der Heide D, Albers MJ (2010) Off-hours admission and mortality in two pediatric intensive care units without 24-h in-house senior staff attendance. Intensive Care Med 36(11):1923–1927. doi:10.1007/s00134-010-2020-4
  6. Latour JM, van Goudoever JB, Duivenvoorden HJ, van Dam NA, Dullaart E, Albers MJ, Verlaat CW, van Vught EM, van Heerde M, Hazelzet JA (2009) Perceptions of parents on satisfaction with care in the pediatric intensive care unit: the EMPATHIC study. Intensive Care Med 35(6):1082–1089. doi:10.1007/s00134-009-1491-7
  7. Visser IHE, PICE-studygroup (2008) Dutch pediatric intensive care evaluation. PICE Report 2003–2005. ISBN 9789078022039. http://www.pice.nl/eng/gegevens-eng.htm. Accessed 10 Apr 2012
  8. Gemke RJ, Bonsel GJ (1995) Comparative assessment of pediatric intensive care: a national multicenter study. Pediatric Intensive Care Assessment of Outcome (PICASSO) study group. Crit Care Med 23(2):238–245
    • View reference on PubMed
    • View reference on publisher's website
  9. Pollack MM, Patel KM, Ruttimann UE (1996) PRISM III: an updated pediatric risk of mortality score. Crit Care Med 24(5):743–752
    • View reference on PubMed
    • View reference on publisher's website
  10. Shann F, Pearson G, Slater A, Wilkinson K (1997) Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med 23(2):201–207
  11. Slater A, Shann F, Pearson G (2003) PIM2: a revised version of the paediatric index of mortality. Intensive Care Med 29(2):278–285
    • View reference on PubMed
  12. PICANet (2007) Paediatric intensive care audit network national report 2004–2006 (published June 2007): Universities of Leeds and Leicester. ISBN 9780853162643. http://www.picanet.org.uk/documentation.html. Accessed 6 Oct 2011
  13. Alexander J, Tregea S, Slater A (2008) Report of the Australian and New Zealand paediatric intensive care registry 2006. Australian and New Zealand Intensive Care Society (ANZICS). ISBN 1876980303. http://www.anzics.com.au/anzpicr. Accessed 8 Feb 2013
  14. Alexander J, Tregea S, Slater A (2010) Report of the Australian and New Zealand paediatric intensive care registry 2008. Australian and New Zealand Intensive Care Society (ANZICS). ISBN 1876980699. http://www.anzics.com.au/anzpicr. Accessed 8 Feb 2013
  15. Wolfler A, Silvani P, Musicco M, Salvo I (2007) Pediatric index of mortality 2 score in Italy: a multicenter, prospective, observational study. Intensive Care Med 33(8):1407–1413. doi:10.1007/s00134-007-0694-z
  16. Brady AR, Harrison D, Black S, Jones S, Rowan K, Pearson G, Ratcliffe J, Parry GJ (2006) Assessment and optimization of mortality prediction tools for admissions to pediatric intensive care in the United Kingdom. Pediatrics 117(4):e733–e742
    • View reference on PubMed
    • View reference on publisher's website
  17. Slater A, Shann F (2004) The suitability of the pediatric index of mortality (PIM), PIM2, the pediatric risk of mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatr Crit Care Med 5(5):447–454
    • View reference on PubMed
    • View reference on publisher's website
  18. Steyerberg EW (2009) Clinical prediction models. A practical approach to development, validation, and updating. Springer, New York
  19. Collins GS, Moons KG (2012) Comparing risk prediction models. BMJ 344:e3186
    • View reference on PubMed
    • View reference on publisher's website
  20. Liddell FD (1984) Simple exact analysis of the standardised mortality ratio. J Epidemiol Community Health 38(1):85–88
    • View reference on PubMed
    • View reference on publisher's website
  21. Cox DR (1958) Two further applications of a model for binary regression. Biometrika 45(3/4):562–565
    • View reference on publisher's website
  22. Hanley JAMB (1982) The meaning and use of the area under receiver operating characteristics (ROC) curve. Radiology 143:29–36
    • View reference on PubMed
  23. Slater A, Shann F, McEniery J (2003) The ANZPIC registry diagnostic codes: a system for coding reasons for admitting children to intensive care. Intensive Care Med 29(2):271–277
    • View reference on PubMed
  24. Vergouwe Y, Steyerberg EW, Eijkemans MJ, Habbema JD (2005) Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol 58(5):475–483. doi:10.1016/j.jclinepi.2004.06.017
    • View reference on PubMed
    • View reference on publisher's website
  25. Thukral A, Lodha R, Irshad M, Arora NK (2006) Performance of pediatric risk of mortality (PRISM), pediatric index of mortality (PIM), and PIM2 in a pediatric intensive care unit in a developing country. Pediatr Crit Care Med 7(4):356–361. doi:10.1097/01.PCC.0000227105.20897.89
    • View reference on PubMed
    • View reference on publisher's website
  26. Thorburn K, Baines P (2002) Predicting mortality or recognizing death in meningococcal disease? Pediatric risk of mortality (PRISM) and pediatric index of mortality (PIM). Crit Care Med 30(7):1675–1676
    • View reference on PubMed
    • View reference on publisher's website
  27. Mildh L, Pettila V, Sairanen H, Rautiainen P (2007) Predictive value of paediatric risk of mortality score and risk adjustment for congenital heart surgery score after paediatric open-heart surgery. Interact Cardiovasc Thorac Surg 6(5):628–631. doi:10.1510/icvts.2007.157917
    • View reference on PubMed
    • View reference on publisher's website
  28. Leteurtre S, Leclerc F, Martinot A, Cremer R, Fourier C, Sadik A, Grandbastien B (2001) Can generic scores (pediatric risk of mortality and pediatric index of mortality) replace specific scores in predicting the outcome of presumed meningococcal septic shock in children? Crit Care Med 29(6):1239–1246
    • View reference on PubMed
    • View reference on publisher's website
  29. Czaja AS, Scanlon MC, Kuhn EM, Jeffries HE (2010) Performance of the pediatric index of mortality 2 for pediatric cardiac surgery patients. Pediatr Crit Care Med. doi:10.1097/PCC.0b013e3181e89694
  30. Wang C, Xie G, Cheng B, Du L, Shi L, Tan L, Shu Q, Fang X (2009) Performance of pediatric risk of mortality, pediatric index of mortality and PIM2 in term Chinese neonates. J Trop Pediatr. doi:10.1093/tropej/fmp124
  31. Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD (2004) Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 23(16):2567–2586. doi:10.1002/sim.1844
    • View reference on PubMed
    • View reference on publisher's website
  32. Leteurtre S, Grandbastien B, Leclerc F, Parslow R (2012) International comparison of the performance of the paediatric index of mortality (PIM) 2 score in two national data sets. Intensive Care Med. doi:10.1007/s00134-012-2580-6
    • View reference on PubMed
  33. Gorman M, Sachdeva R, Rice T, Wetzel R (2009) Support for PICUs and the PRISM scoring algorithms. Pediatr Crit Care Med 10(1):143–144. doi:10.1097/PCC.0b013e3181937bf4
    • View reference on PubMed
    • View reference on publisher's website
  34. Tilford JM, Roberson PK, Lensing S, Fiser DH (1998) Differences in pediatric ICU mortality risk over time. Crit Care Med 26(10):1737–1743
    • View reference on PubMed
    • View reference on publisher's website
  35. Kramer AA, Zimmerman JE (2010) A predictive model for the early identification of patients at risk for a prolonged intensive care unit length of stay. BMC Med Inform Decis Mak 10:27. doi:10.1186/1472-6947-10-27
    • View reference on PubMed
    • View reference on publisher's website
  36. Randolph AG (1997) Paediatric index of mortality (PIM): do we need another paediatric mortality prediction score? Intensive Care Med 23(2):141–142
  37. Steyerberg EW, Bleeker SE, Moll HA, Grobbee DE, Moons KG (2003) Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol 56(5):441–447 (pii: S0895435603000477)
    • View reference on PubMed
    • View reference on publisher's website

Sign In

Connect with ICM

Top 5 Articles Editors Picks Supplement