Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.

Estee Y Cramer ORCID logo; Evan L Ray ORCID logo; Velma K Lopez ORCID logo; Johannes Bracher ORCID logo; AndreaBrennen; Alvaro JCastro Rivadeneira; AaronGerding; Tilmann Gneiting ORCID logo; Katie HHouse; YuxinHuang; +285 more... DasuniJayawardena; Abdul HKanji; AyushKhandelwal; KhoaLe; AnjaMühlemann; Jarad Niemi ORCID logo; ApurvShah; ArianeStark; YijinWang; NutchaWattanachit; Martha WZorn; YouyangGu; SansiddhJain; NayanaBannur; AyushDeva; MihirKulkarni; SrujanaMerugu; AlpanRaval; SiddhantShingi; AvtanshTiwari; Jerome White ORCID logo; Neil FAbernethy; Spencer Woody ORCID logo; MaytalDahan; Spencer Fox ORCID logo; Kelly Gaither ORCID logo; MichaelLachmann; Lauren Ancel Meyers ORCID logo; James GScott; Mauricio Tec ORCID logo; AjiteshSrivastava; Glover E George ORCID logo; Jeffrey C Cegan ORCID logo; Ian DDettwiller; William PEngland; Matthew WFarthing; Robert H Hunter ORCID logo; Brandon Lafferty ORCID logo; IgorLinkov; Michael L Mayo ORCID logo; Matthew DParno; Michael A Rowland ORCID logo; Benjamin DTrump; YanliZhang-James; Samuel Chen ORCID logo; Stephen VFaraone; JonathanHess; Christopher PMorley; Asif Salekin ORCID logo; DongliangWang; Sabrina M Corsetti ORCID logo; Thomas MBaer; Marisa CEisenberg; Karl Falb ORCID logo; Yitao Huang ORCID logo; Emily TMartin; EllaMcCauley; Robert LMyers; TomSchwarz; Daniel Sheldon ORCID logo; Graham CaseyGibson; RoseYu; LiyaoGao; YianMa; DongxiaWu; XifengYan; XiaoyongJin; Yu-XiangWang; YangQuanChen; Lihong Guo ORCID logo; YantingZhao; Quanquan Gu ORCID logo; JinghuiChen; LingxiaoWang; Pan Xu ORCID logo; WeitongZhang; DifanZou; HannahBiegel; Joceline Lega ORCID logo; Steve McConnell ORCID logo; VPNagraj; Stephanie LGuertin; ChristopherHulme-Lowe; Stephen D Turner ORCID logo; Yunfeng Shi ORCID logo; XuegangBan; Robert Walraven ORCID logo; Qi-JunHong; StanleyKong; Axel van de Walle ORCID logo; James A Turtle ORCID logo; Michal Ben-Nun ORCID logo; Steven Riley ORCID logo; PeteRiley; Ugur Koyluoglu ORCID logo; DavidDesRoches; PedroForli; BruceHamory; ChristinaKyriakides; HelenLeis; JohnMilliken; MichaelMoloney; JamesMorgan; NinadNirgudkar; GokceOzcan; NoahPiwonka; MattRavi; ChrisSchrader; ElizabethShakhnovich; DanielSiegel; RyanSpatz; ChrisStiefeling; BarrieWilkinson; AlexanderWong; Sean Cavany ORCID logo; Guido España ORCID logo; Sean Moore ORCID logo; Rachel Oidtman ORCID logo; Alex Perkins ORCID logo; David Kraus ORCID logo; AndreaKraus; ZhifengGao; JiangBian; Wei Cao ORCID logo; Juan Lavista Ferres ORCID logo; ChaozhuoLi; Tie-YanLiu; XingXie; ShunZhang; ShunZheng; Alessandro Vespignani ORCID logo; MatteoChinazzi; Jessica TDavis; KunpengMu; AnaPastore Y Piontti; XinyueXiong; AndrewZheng; JackieBaek; VivekFarias; AndreeaGeorgescu; RetsefLevi; Deeksha Sinha ORCID logo; JoshuaWilde; Georgia Perakis ORCID logo; Mohammed Amine Bennouna ORCID logo; DavidNze-Ndong; DivyaSinghvi; Ioannis Spantidakis ORCID logo; LeannThayaparan; Asterios Tsiourvas ORCID logo; Arnab Sarker ORCID logo; Ali Jadbabaie ORCID logo; Devavrat Shah ORCID logo; NicolasDella Penna; Leo A Celi ORCID logo; SakethSundar; RussWolfinger; Dave Osthus ORCID logo; LaurenCastro; Geoffrey Fairchild ORCID logo; IsaacMichaud; DeanKarlen; MattKinsey; Luke C Mullany ORCID logo; Kaitlin Rainwater-Lovett ORCID logo; LaurenShin; KatharineTallaksen; ShelbyWilson; Elizabeth C Lee ORCID logo; Juan Dent ORCID logo; Kyra HGrantz; Alison L Hill ORCID logo; JoshuaKaminsky; KathrynKaminsky; Lindsay T Keegan ORCID logo; Stephen ALauer; Joseph C Lemaitre ORCID logo; JustinLessler; Hannah RMeredith; JavierPerez-Saez; SamShah; Claire PSmith; Shaun A Truelove ORCID logo; Josh Wills ORCID logo; MaximilianMarshall; LaurenGardner; KristenNixon; John CBurant; LilyWang; Lei Gao ORCID logo; Zhiling Gu ORCID logo; MyungjinKim; XinyiLi; GuannanWang; YueyingWang; Shan Yu ORCID logo; Robert CReiner; RyanBarber; EmmanuelaGakidou; Simon I Hay ORCID logo; SteveLim; Chris Murray ORCID logo; DavidPigott; Heidi LGurung; PrasithBaccam; Steven A Stage ORCID logo; Bradley TSuchoski; B Aditya Prakash ORCID logo; BijayaAdhikari; JiamingCui; Alexander Rodríguez ORCID logo; AnikaTabassum; Jiajia Xie ORCID logo; Pinar Keskinocak ORCID logo; JohnAsplund; Arden Baxter ORCID logo; Buse Eylul Oruc ORCID logo; NicoletaSerban; Sercan OArik; MikeDusenberry; ArkadyEpshteyn; ElliKanal; Long TLe; Chun-LiangLi; TomasPfister; DarioSava; Rajarishi Sinha ORCID logo; ThomasTsai; Nate Yoder ORCID logo; JinsungYoon; Leyou Zhang ORCID logo; SamAbbott; Nikos IBosse; Sebastian Funk ORCID logo; JoelHellewell; Sophie R Meakin ORCID logo; Katharine Sherratt ORCID logo; MingyuanZhou; RahiKalantari; Teresa K Yamana ORCID logo; Sen Pei ORCID logo; Jeffrey Shaman ORCID logo; Michael L Li ORCID logo; Dimitris Bertsimas ORCID logo; Omar Skali Lami ORCID logo; Saksham Soni ORCID logo; Hamza Tazi Bouardi ORCID logo; TurgayAyer; MadelineAdee; JagpreetChhatwal; Ozden ODalgic; Mary ALadd; Benjamin PLinas; PeterMueller; JadeXiao; Yuanjia Wang ORCID logo; QinxiaWang; ShanghongXie; DonglinZeng; AldenGreen; JacobBien; LoganBrooks; Addison JHu; MariaJahja; Daniel McDonald ORCID logo; BalasubramanianNarasimhan; Collin Politsch ORCID logo; Samyak Rajanala ORCID logo; Aaron Rumack ORCID logo; NoahSimon; Ryan J Tibshirani ORCID logo; RobTibshirani; ValerieVentura; LarryWasserman; Eamon BO'Dea; John M Drake ORCID logo; RobertPagano; Quoc TTran; Lam Si Tung Ho ORCID logo; HuongHuynh; Jo WWalker; Rachel B Slayton ORCID logo; Michael A Johansson ORCID logo; Matthew Biggerstaff ORCID logo; Nicholas G Reich ORCID logo; (2022) Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States. Proceedings of the National Academy of Sciences, 119 (15). e2113561119-. ISSN 0027-8424 DOI: 10.1073/pnas.2113561119
Copy

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.



picture_as_pdf
Cramer_etal_2021_Evaluation-of-individual-and-ensemble.pdf
subject
Published Version
Available under Creative Commons: 4.0

View Download

Explore Further

Read more research from the creator(s):

Find work associated with the faculties and division(s):

Find work associated with the research centre(s):

Find work from this publication: