Data analysis shows ICU scores accurate in predicting risk of death

Using clinical data from more than 200 hospital intensive care units, Philips Healthcare has shown that three ICU risk scores—designed for different purposes—performed well as a marker of severity of illness at admission and throughout the ICU stay.

The analysis of de-identified data from more than 560,000 ICU patient stays contributed by 333 ICUs, covering almost 39 million patient-hours of ICU care, reveal that it is possible for risk models to perform well even when deployed for uses other than what they were originally intended.

Three risk scores were evaluated as predictors of mortality risk: Acute Physiology and Chronic Health Evaluation IV (APACHE), designed to estimate the risk of death on admission to the ICU; Discharge Readiness Score (DRS), designed to assist ICU discharge decisions by estimating the risk of death in the first 48 hours after the patient leaves the ICU; and Sequential Organ Failure Assessment (SOFA), designed to assess organ failure risk in patients with sepsis.

“Each was developed for a different purpose,” says Omar Badawi, director of clinical analytics and reporting for the eICU program at Philips Health Systems. “What we find is there are a lot of risk models that are used in the ICU, and as soon as they’re available for one thing, clinicians tend to want to use them for other purposes. There’s a long history of algorithms being repurposed.”

Also See: MIT researchers use machine learning to predict ICU interventions

Results of the non-intervention cohort study, published in the March issue of the journal Critical Care Medicine, report that APACHE, DRS and SOFA all demonstrated good accuracy in predicting the risk of death in the ICU and the risk of death within 24 hours. However, of the three scores, DRS had the highest predictive value.

“These findings validate the use of these models on a population level for continuous risk adjustment in the ICU, although APACHE and SOFA appear slower to respond to improvements in patient status than DRS, and DRS may reflect physiologic improvement from interventions, potentially underestimating risk,” conclude the study’s authors.

While they contend that these findings suggest that it is possible to repurpose risk models for use outside of their original design, the authors insist that this should be done only after thorough validation and understanding of how the design of each risk model may influence performance in different ICU settings.

“Risk models provide objective data regarding patient status but users must combine their clinical judgment with an understanding of the model design in order to interpret the score for specific patients,” cautions Badawi.