CaltechAUTHORS
  A Caltech Library Service

Training physics-based machine-learning parameterizations with gradient-free ensemble Kalman methods

Lopez-Gomez, Ignacio and Christopoulos, Costa and Ervik, Haakon Ludvig Langeland and Dunbar, Oliver R. A. and Cohen, Yair and Schneider, Tapio (2022) Training physics-based machine-learning parameterizations with gradient-free ensemble Kalman methods. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20220331-531674000

[img] PDF (22 Jun 2022) - Submitted Version
Creative Commons Attribution.

1MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20220331-531674000

Abstract

Most machine learning applications in Earth system modeling currently rely on gradient-based supervised learning. This imposes stringent constraints on the nature of the data used for training (typically, residual time tendencies are needed), and it complicates learning about the interactions between machine-learned parameterizations and other components of an Earth system model. Approaching learning about process-based parameterizations as an inverse problem resolves many of these issues, since it allows parameterizations to be trained with partial observations or statistics that directly relate to quantities of interest in long-term climate projections. Here we demonstrate the effectiveness of Kalman inversion methods in treating learning about parameterizations as an inverse problem. We consider two different algorithms: unscented and ensemble Kalman inversion. Both methods involve highly parallelizable forward model evaluations, converge exponentially fast, and do not require gradient computations. In addition, unscented Kalman inversion provides a measure of parameter uncertainty. We illustrate how training parameterizations can be posed as a regularized inverse problem and solved by ensemble Kalman methods through the calibration of an eddy-diffusivity mass-flux scheme for subgrid-scale turbulence and convection, using data generated by large-eddy simulations. We find the algorithms amenable to batching strategies, robust to noise and model failures, and efficient in the calibration of hybrid parameterizations that can include empirical closures and neural networks.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
https://doi.org/10.1002/essoar.10510937.2DOIDiscussion Paper
https://doi.org/10.5281/zenodo.6382968DOISoftware
https://doi..org/10.5281/zenodo.6392397DOISoftware
https://doi.org/10.5281/zenodo.6382865DOISoftware
https://doi.org/10.22002/D1.20052DOIData
ORCID:
AuthorORCID
Lopez-Gomez, Ignacio0000-0002-7255-5895
Christopoulos, Costa0000-0002-8552-465X
Ervik, Haakon Ludvig Langeland0000-0003-2912-5774
Dunbar, Oliver R. A.0000-0001-7374-0382
Cohen, Yair0000-0002-9615-2476
Schneider, Tapio0000-0001-5687-2287
Additional Information:License: Attribution 4.0 International. Submission History: [v1] Sat, 26 Mar 2022 17:32:48 (883.1 KB); [v2] Wed, 22 Jun 2022 16:15:12 (919.4 KB). We thank Daniel Z. Huang and Zhaoyi Shen for insightful discussions, and Julien Brajard and an anonymous reviewer for prompting a clearer and more precise formulation of the problem and methods discussed in this study. I.L. was supported by a fellowship from the Resnick Sustainability Institute at Caltech, and an Amazon AI4Science fellowship. H.L.L.E was supported by an Aker scholarship and a Fulbright fellowship. This research was additionally supported by the generosity of Eric and Wendy Schmidt by recommendation of the Schmidt Futures program, by the National Science Foundation (grant AGS-1835860), by the Defense Advanced Research Projects Agency (Agreement No. HR00112290030), and by the Heising-Simons Foundation. Part of this research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. The software package implementing ensemble Kalman methods can be accessed at https://doi.org/10.5281/zenodo.6382968, the one implementing the EDMF scheme at https://doi..org/10.5281/zenodo.6392397, and the software used to calibrate the EDMF scheme may be accessed at https://doi.org/10.5281/zenodo.6382865. The data from Shen et al. (2022) used for model training is available at https://doi.org/10.22002/D1.20052.
Group:Resnick Sustainability Institute
Funders:
Funding AgencyGrant Number
Resnick Sustainability InstituteUNSPECIFIED
Amazon AI4Science FellowshipUNSPECIFIED
Aker Scholarship FoundationUNSPECIFIED
Fulbright FoundationUNSPECIFIED
Schmidt Futures ProgramUNSPECIFIED
NSFAGS-1835860
Defense Advanced Research Projects Agency (DARPA)HR00112290030
Heising-Simons FoundationUNSPECIFIED
NASA/JPL/CaltechUNSPECIFIED
DOI:10.1002/essoar.10510937.2
Record Number:CaltechAUTHORS:20220331-531674000
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20220331-531674000
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:114134
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:01 Apr 2022 18:33
Last Modified:12 Jul 2022 20:53

Repository Staff Only: item control page