RWJBarnabas Health and Rutgers Researchers Find AI Tool Helps Detect Patients at Risk Earlier

Findings from a study of more than 23,000 high-risk patients across 11 hospitals may help reduce deaths

Researchers from RWJBarnabas Health and Rutgers Robert Wood Johnson Medical School found that an artificial intelligence (AI)-enabled early warning system helped identify hospitalized patients at risk of rapid clinical decline sooner, contributing to fewer deaths among high-risk patients.

In the study, published in NEJM AI, a journal from the New England Journal of Medicine group, researchers evaluated outcomes among 23,132 high-risk patients across 11 RWJBarnabas Health hospitals. Deaths among high-risk patients fell from 23.1% to 18.6% following implementation of the AI-enabled early warning system, representing an 18% reduction in the risk-adjusted odds of in-hospital death.

"Our goal was to identify patients earlier, before they reached a point where intervention becomes much more difficult," said Thomas Nahass, an intensive care and clinical informatics physician at RWJBarnabas Health and Rutgers Robert Wood Johnson Medical School and lead author of the study. "If we can get a critical care eye on the patient sooner, we can change the course of their outcome."

Our goal was to identify patients earlier, before they reached a point where intervention becomes much more difficult. If we can get a critical care eye on the patient sooner, we can change the course of their outcome.

Thomas Nahass

Intensive Care & Clinical Informatics Physician, RWJBarnabas Health & Robert Wood Johnson Medical School

Hospitalized patients can deteriorate quickly, often before obvious warning signs become apparent. Researchers evaluated the Epic Deterioration Index (EDI), a proprietary, machine-learning tool that continuously analyzes information already captured in the electronic health record, including vital signs, laboratory results, nursing assessments and age, to identify patients at increased risk of serious clinical decline. The system recalculates risk scores every 15 minutes and automatically alerts rapid response teams when patients reach the highest-risk category.

Before evaluating the technology in this study, RWJBarnabas Health and Rutgers researchers spent several years developing and implementing a systemwide approach to using the EDI across its hospitals. RWJBarnabas Health first integrated the tool into its electronic health record at its quaternary academic medical center, Robert Wood Johnson University Hospital, to pilot and refine it, including how and when alerts were delivered, established automatic notifications to rapid response teams, trained clinicians on its use and continuously monitored its performance. Researchers from Rutgers then partnered with RWJBarnabas Health to evaluate the impact of this approach in real-world clinical practice and rapidly rolled out the platform for the other 10 hospitals.

When patients reached the highest-risk threshold, automated notifications were sent directly to hospital rapid response teams, enabling critical care specialists to quickly assess patients and determine whether additional interventions were needed. Following implementation, rapid response team activations among high-risk patients increased from 25.3% of hospital stays to 37.5%.

The study evaluated outcomes among high-risk adult patients receiving care at academic medical centers, community teaching hospitals and community hospitals throughout the RWJBarnabas Health system. Despite the increase in rapid response evaluations, transfers to intensive care units didn't significantly increase, while mortality rates declined sharply.

"Every minute matters when a patient's condition begins to worsen," said Andy Anderson, chief medical and quality officer, RWJBarnabas Health and a co-author of the study. "This study demonstrates how AI-enabled tools, when paired with experienced clinical teams can help us identify patients at risk sooner and deliver the right care at the right time. These findings highlight the potential for innovation to improve quality, safety and outcomes for the patients we serve."

This study demonstrates how AI-enabled tools, when paired with experienced clinical teams can help us identify patients at risk sooner and deliver the right care at the right time.

Andy Anderson

Chief Medical & Quality Officer, RWJBarnabas Health

"This is what an integrated academic health system is for," said Stephen P. O'Mahony, senior vice president and chief medical information office at RWJBarnabas Health and a senior author of the study. "We combined Rutgers methodological rigor with the operational reach of 11 RWJBarnabas hospitals. The mortality benefit was not produced by an algorithm but by the partnership around the algorithm."

Researchers said the mortality benefit likely resulted from a combination of factors, including staff education, enhanced clinical awareness, electronic health record alerts and automated rapid response team notifications working together as a coordinated systemwide approach. Because the study evaluated the Epic Deterioration Index, a tool already integrated into Epic Systems Corp.'s electronic health records system, the findings may have implications for hospitals nationwide seeking to improve patient outcomes. Researchers are now evaluating the next phase of the initiative, which focuses on identifying patients whose risk scores are rising rapidly in hopes of enabling even earlier intervention.

Along with Nahass and O'Mahony and, authors include: co-first author Joseph S. Hanna, Rutgers Robert Wood Johnson Medical School and Robert Wood Johnson University Hospital; Nancy Liu and Jason Roy, Rutgers School of Public Health; Nicole Martinez, RWJBarnabas Health; Ethan A. Halm, Rutgers Robert Wood Johnson Medical School; Amy Rockman, Rutgers Biomedical and Health Sciences; Christopher Gilligan, Robert Wood Johnson University Hospital; Amy P. Murtha, Rutgers Robert Wood Johnson Medical School and Vicente Gracias, Rutgers Biomedical and Health Sciences.

Explore more of the ways Rutgers research is shaping the future.