A Caltech Library Service

Real-Time Electronic Health Record Mortality Prediction During the COVID-19 Pandemic: A Prospective Cohort Study

Sottile, Peter D. and Albers, David and DeWitt, Peter E. and Russell, Seth and Stroh, J. N. and Kao, David P. and Adrian, Bonnie and Levine, Matthew E. and Mooney, Ryan and Larchick, Lenny and Kutner, Jean S. and Wynia, Matthew K. and Glasheen, Jeffrey J. and Bennett, Tellen D. (2021) Real-Time Electronic Health Record Mortality Prediction During the COVID-19 Pandemic: A Prospective Cohort Study. . (Unpublished)

[img] PDF - Submitted Version
See Usage Policy.

[img] MS Word - Supplemental Material
See Usage Policy.


Use this Persistent URL to link to this item:


Background: The SARS-CoV-2 virus has infected millions of people, overwhelming critical care resources in some regions. Many plans for rationing critical care resources during crises are based on the Sequential Organ Failure Assessment (SOFA) score. The COVID-19 pandemic created an emergent need to develop and validate a novel electronic health record (EHR)-computable tool to predict mortality. Research Questions: To rapidly develop, validate, and implement a novel real-time mortality score for the COVID-19 pandemic that improves upon SOFA. Study Design and Methods: We conducted a prospective cohort study of a regional health system with 12 hospitals in Colorado between March 2020 and July 2020. All patients >14 years old hospitalized during the study period without a do not resuscitate order were included. Patients were stratified by the diagnosis of COVID-19. From this cohort, we developed and validated a model using stacked generalization to predict mortality using data widely available in the EHR by combining five previously validated scores and additional novel variables reported to be associated with COVID-19-specific mortality. We compared the area under the receiver operator curve (AUROC) for the new model to the SOFA score and the Charlson Comorbidity Index. Results: We prospectively analyzed 27,296 encounters, of which 1,358 (5.0%) were positive for SARS-CoV-2, 4,494 (16.5%) included intensive care unit (ICU)-level care, 1,480 (5.4%) included invasive mechanical ventilation, and 717 (2.6%) ended in death. The Charlson Comorbidity Index and SOFA scores predicted overall mortality with an AUROC of 0.72 and 0.90, respectively. Our novel score predicted overall mortality with AUROC 0.94. In the subset of patients with COVID-19, we predicted mortality with AUROC 0.90, whereas SOFA had AUROC of 0.85. Interpretation: We developed and validated an accurate, in-hospital mortality prediction score in a live EHR for automatic and continuous calculation using a novel model, that improved upon SOFA. Study Question: Can we improve upon the SOFA score for real-time mortality prediction during the COVID-19 pandemic by leveraging electronic health record (EHR) data? Results: We rapidly developed and implemented a novel yet SOFA-anchored mortality model across 12 hospitals and conducted a prospective cohort study of 27,296 adult hospitalizations, 1,358 (5.0%) of which were positive for SARS-CoV-2. The Charlson Comorbidity Index and SOFA scores predicted all-cause mortality with AUROCs of 0.72 and 0.90, respectively. Our novel score predicted mortality with AUROC 0.94. Interpretation: A novel EHR-based mortality score can be rapidly implemented to better predict patient outcomes during an evolving pandemic.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Albers, David0000-0002-5369-526X
Levine, Matthew E.0000-0002-5627-3169
Bennett, Tellen D.0000-0003-1483-4236
Alternate Title:Real-Time Mortality Prediction During COVID-19
Additional Information:The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. This version posted January 15, 2021. PS is supported by NIH K23 HL 145001, DA and ML by NIH R01 LM012734, DK by NIH K08 HL125725, TB by NIH UL1 TR002535 and UL1 TR002535 03S2. Data Availability: The data referred to in the manuscript are the property of UCHealth and we are unable to share them. Author Declarations: I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes. The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The Colorado Multiple Institutional Review Board (COMIRB) reviewed this protocol (#20-0995) and approved it for exemption. All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived. Yes. I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes. I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes. All authors do not have a conflict of interests. Competing Interest Statement: BA, RM, LL, and JK, and JG are employees of UCHealth. No other competing interests to declare.
Funding AgencyGrant Number
NIHK23 HL145001
NIHR01 LM012734
NIHK08 HL125725
NIHUL1 TR002535
NIHUL1 TR002535 03S2
Record Number:CaltechAUTHORS:20210120-124914573
Persistent URL:
Official Citation:Real-Time Electronic Health Record Mortality Prediction During the COVID-19 Pandemic: A Prospective Cohort Study. Peter D. Sottile, David Albers, Peter E. DeWitt, Seth Russell, J.N. Stroh, David P. Kao, Bonnie Adrian, Matthew E. Levine, Ryan Mooney, Lenny Larchick, Jean S. Kutner, Matthew K. Wynia, Jeffrey J. Glasheen, Tellen D. Bennett. medRxiv 2021.01.14.21249793; doi:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:107593
Deposited By: Tony Diaz
Deposited On:20 Jan 2021 23:22
Last Modified:02 Feb 2021 22:59

Repository Staff Only: item control page