Log in | Register

International comparison of the performance of the paediatric index of mortality (PIM) 2 score in two national data sets

Stéphane Leteurtre| Bruno Grandbastien| Francis Leclerc| Roger Parslow
Pediatric Original
Volume 38, Issue 8 / August , 2012

Pages 1372 - 1380

Abstract

Objectives

To test the performance of PIM2 in French-speaking (FS) paediatric intensive care units (PICUs) and its relative performance when recalibrated using data from FS and Great Britain (GB) PICUs of different size.

Methods

Consecutive admissions to 15 FS (n = 5,602) and 31 GB PICUs (n = 20,693) from June 2006 to October 2007 were included. The recalibrated PIM2 were applied to PICUs of different size within the FS and GB PICUs and between the two groups. PICU size was defined using number of admissions/month. Discrimination and calibration were evaluated using the area under the ROC curve (AUC) and the goodness-of-fit test, respectively. Logistic regression, funnel plots and standardized W scores were performed in the two groups and between different PICU sizes.

Results

In FS PICUs, the original PIM2 had good discrimination (AUC = 0.85) and moderate calibration (p = 0.07). The recalibrated PIM2 scores had good calibration in FS (p = 0.33) and moderate calibration in GB (p = 0.06). Calibration was poor when the recalibrated FS PIM2 was applied to GB (p = 0.02) but good when the GB recalibration was applied to the FS (p = 0.36). Using the original PIM2 coefficients, calibration was poor in large units in both groups but improved following recalibration. There were no effects of PICU size on risk-adjusted mortality in GB and a significant effect in the FS PICUs with a minimum risk-adjusted mortality at about 35 admissions/month.

Conclusion

The PIM2 score was valid in the FS population. The recalibration based on GB data could be applied to FS PICUs. Such recalibration may facilitate comparisons between countries.

Keywords

References

  1. Lacroix J, Cotting J (2005) Severity of illness and organ dysfunction scoring in children. Pediatr Crit Care Med 6:S126–S134
    • View reference on PubMed
    • View reference on publisher's website
  2. Martinot A, Leteurtre S, Grandbastien B, Duhamel A, Leclerc F (1997) Characteristics of patients and use of resource in French pediatric intensive care units. Arch Pediatr 4:730–736
    • View reference on PubMed
    • View reference on publisher's website
  3. Slater A, Shann F (2004) The suitability of the pediatric index of mortality (PIM), PIM2, the pediatric risk of mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatr Crit Care Med 5:447–454
    • View reference on PubMed
    • View reference on publisher's website
  4. Wunsch H, Rowan KM, Angus DC (2007) International comparisons in critical care: a necessity and challenge. Curr Opin Crit Care 13:725–731
    • View reference on PubMed
    • View reference on publisher's website
  5. LaRovere JM, Jeffries HE, Sachdeva RC, Rice TB, Wetzel RC, Cooper DS, Bird GL, Ghanayem NS, Checchia PA, Chang AC, Wessel DL (2008) Databases for assessing the outcomes of the treatment of patients with congenital and paediatric cardiac disease: the perspective of critical care. Cardiol Young 18(Suppl 2):130–136
    • View reference on PubMed
    • View reference on publisher's website
  6. Parslow RC, Tasker RC, Draper ES, Parry GJ, Jones S, Chater T, Thiru K, McKinney PA (2009) Epidemiology of critically ill children in England and Wales: incidence, mortality, deprivation and ethnicity. Arch Dis Child 94:210–215
    • View reference on PubMed
    • View reference on publisher's website
  7. Slater A, Shann F, McEniery J (2003) The ANZPIC registry diagnostic codes: a system for coding reasons for admitting children to intensive care. Intensive Care Med 29:271–277
    • View reference on PubMed
  8. Pollack MM, Patel KM, Ruttimann UE (1996) PRISM III: an updated pediatric risk of mortality score. Crit Care Med 24:743–752
    • View reference on PubMed
    • View reference on publisher's website
  9. Slater A, Shann F, Pearson G (2003) PIM2: a revised version of the paediatric index of mortality. Intensive Care Med 29:278–285
    • View reference on PubMed
  10. Brady AR, Harrison D, Black S, Jones S, Rowan K, Pearson G, Ratcliffe J, Parry GJ (2006) Assessment and optimization of mortality prediction tools for admissions to pediatric intensive care in the United Kingdom. Pediatrics 117:e733–e742
    • View reference on PubMed
    • View reference on publisher's website
  11. Chase JG, Shaw GM (2012) How standard is the “S” in SMR? Intensive Care Med 38:1–3
  12. Minne L, Eslami S, de Keizer N, de Jonge E, de Rooij SE, Abu-Hanna A (2012) Effect of changes over time in the performance of a customized SAPS-II model on the quality of care assessment. Intensive Care Med 38:40–46
  13. Vincent JL, Moreno R (2010) Clinical review: scoring systems in the critically ill. Crit Care 14:207
    • View reference on PubMed
    • View reference on publisher's website
  14. Wolfler A, Silvani P, Musicco M, Salvo I (2007) Pediatric index of mortality 2 score in Italy: a multicenter, prospective, observational study. Intensive Care Med 33:1407–1413
  15. Tilford JM, Simpson PM, Green JW, Lensing S, Fiser DH (2000) Volume-outcome relationships in pediatric intensive care units. Pediatrics 106:289–294
    • View reference on PubMed
    • View reference on publisher's website
  16. Cogo PE, Poole D, Codazzi D, Boniotti C, Capretta A, Langer M, Luciani D, Rossi C, Bertolini G (2010) Outcome of children admitted to adult intensive care units in Italy between 2003 and 2007. Intensive Care Med 36:1403–1409
  17. Pollack MM (2007) Pediatric intensive care quality factors. J Trauma 63:S143–S145
    • View reference on PubMed
    • View reference on publisher's website
  18. Marcin JP, Song J, Leigh JP (2005) The impact of pediatric intensive care unit volume on mortality: a hierarchical instrumental variable analysis. Pediatr Crit Care Med 6:136–141
    • View reference on PubMed
    • View reference on publisher's website
  19. Pollack MM, Cuerdon TT, Patel KM, Ruttimann UE, Getson PR, Levetown M (1994) Impact of quality-of-care factors on pediatric intensive care unit mortality. JAMA 272:941–946
    • View reference on PubMed
    • View reference on publisher's website
  20. Ruttimann UE, Patel KM, Pollack MM (2000) Relevance of diagnostic diversity and patient volumes for quality and length of stay in pediatric intensive care units. Pediatr Crit Care Med 1:133–139
    • View reference on PubMed
    • View reference on publisher's website
  21. Shahian DM, Normand SL (2008) Comparison of “risk-adjusted” hospital outcomes. Circulation 117:1955–1963
    • View reference on PubMed
    • View reference on publisher's website
  22. Cevenini G, Barbini P (2010) A bootstrap approach for assessing the uncertainty of outcome probabilities when using a scoring system. BMC Med Inform Decis Mak 10:45
    • View reference on PubMed
    • View reference on publisher's website
  23. Lemeshow S, Hosmer DW Jr (1982) A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 115:92–106
    • View reference on PubMed
  24. Hollis S, Yates DW, Woodford M, Foster P (1995) Standardized comparison of performance indicators in trauma: a new approach to case-mix variation. J Trauma 38:763–766
    • View reference on PubMed
    • View reference on publisher's website
  25. Glance LG, Osler T, Shinozaki T (2000) Effect of varying the case mix on the standardized mortality ratio and W statistic: a simulation study. Chest 117:1112–1117
    • View reference on PubMed
    • View reference on publisher's website
  26. Ashley A, Draper E, Fleming T, Lamming C, McKinney P, McShane P, Parslow R, Skinner S, Thiru K (2009) National Report of the Paediatric Intensive Care Audit Network 2006–2008. Leeds. http://www.picanet.org.uk/Documents/General/Annual%20report_2009/PICANet%20National%20Report%202006%20-%202008_new.pdf. Accessed 12 Nov 2011
  27. Eulmesekian PG, Perez A, Minces PG, Ferrero H (2007) Validation of pediatric index of mortality 2 (PIM2) in a single pediatric intensive care unit of Argentina. Pediatr Crit Care Med 8:54–57
    • View reference on PubMed
    • View reference on publisher's website
  28. Imamura T, Nakagawa S, Goldman RD, Fujiwara T (2012) Validation of pediatric index of mortality 2 (PIM2) in a single pediatric intensive care unit in Japan. Intensive Care Med 38:649–654
  29. Leteurtre S, Martinot A, Duhamel A, Proulx F, Grandbastien B, Cotting J, Gottesman R, Joffe A, Pfenninger J, Hubert P, Lacroix J, Leclerc F (2003) Validation of the paediatric logistic organ dysfunction (PELOD) score: prospective, observational, multicentre study. Lancet 362:192–197
    • View reference on PubMed
    • View reference on publisher's website
  30. Metnitz B, Schaden E, Moreno R, Le Gall JR, Bauer P, Metnitz PG (2009) Austrian validation and customization of the SAPS 3 admission score. Intensive Care Med 35:616–622
  31. Bakhshi-Raiez F, Peek N, Bosman RJ, de Jonge E, de Keizer NF (2007) The impact of different prognostic models and their customization on institutional comparison of intensive care units. Crit Care Med 35:2553–2560
    • View reference on PubMed
    • View reference on publisher's website
  32. Gregory CJ, Nasrollahzadeh F, Dharmar M, Parsapour K, Marcin JP (2008) Comparison of critically ill and injured children transferred from referring hospitals versus in-house admissions. Pediatrics 121:e906–e911
    • View reference on PubMed
    • View reference on publisher's website
  33. Alexander J, Tregea S, Slater A (2010) Report of the Australian and New Zealand Paediatric Intensive Care Registry 2009. Available at http://www.anzics.com.au/core/anzpicr?start=2. Accessed 10 Sept 2011
  34. Straney LD, Clements A, Alexander J, Slater A (2010) Measuring efficiency in Australian and New Zealand paediatric intensive care units. Intensive Care Med 36:1410–1416
  35. Haub C (2010) 2010 world population data sheet. Population Reference Bureau, Washington, DC. http://www.prb.org/pdf10/10wpds_eng.pdf. Accessed 4 May 2011
  36. Beyene J, Atenafu EG, Hamid JS, To T, Sung L (2009) Determining relative importance of variables in developing and validating predictive models. BMC Med Res Methodol 9:64
    • View reference on PubMed
    • View reference on publisher's website
  37. Murphy-Filkins R, Teres D, Lemeshow S, Hosmer DW (1996) Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. Crit Care Med 24:1968–1973
    • View reference on PubMed
    • View reference on publisher's website
  38. Metnitz B, Metnitz PG, Bauer P, Valentin A (2009) Patient volume affects outcome in critically ill patients. Wien Klin Wochenschr 121:34–40
    • View reference on PubMed
    • View reference on publisher's website
  39. Glance LG, Szalados JE (2002) Benchmarking in critical care: the road ahead. Chest 121:326–328
    • View reference on PubMed
    • View reference on publisher's website
  40. Macrae D (2009) Conducting clinical trials in pediatrics. Crit Care Med 37:S136–S139
    • View reference on PubMed
    • View reference on publisher's website
  41. Spiegelhalter D (2002) Funnel plots for institutional comparison. Qual Saf Health Care 11:390–391
    • View reference on PubMed
    • View reference on publisher's website

Sign In

Connect with ICM

Top 5 Articles Editors Picks Supplement