A Caltech Library Service

Ensemble Inference Methods for Models With Noisy and Expensive Likelihoods

Dunbar, Oliver R. A. and Duncan, Andrew B. and Stuart, Andrew M. and Wolfram, Marie-Therese (2022) Ensemble Inference Methods for Models With Noisy and Expensive Likelihoods. SIAM Journal on Applied Dynamical Systems, 21 (2). pp. 1539-1572. ISSN 1536-0040. doi:10.1137/21M1410853.

[img] PDF - Published Version
See Usage Policy.

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


The increasing availability of data presents an opportunity to calibrate unknown parameters which appear in complex models of phenomena in the biomedical, physical, and social sciences. However, model complexity often leads to parameter-to-data maps which are expensive to evaluate and are only available through noisy approximations. This paper is concerned with the use of interacting particle systems for the solution of the resulting inverse problems for parameters. Of particular interest is the case where the available forward model evaluations are subject to rapid fluctuations, in parameter space, superimposed on the smoothly varying large-scale parametric structure of interest. A motivating example from climate science is presented, and ensemble Kalman methods (which do not use the derivative of the parameter-to-data map) are shown, empirically, to perform well. Multiscale analysis is then used to analyze the behavior of interacting particle system algorithms when rapid fluctuations, which we refer to as noise, pollute the large-scale parametric dependence of the parameter-to-data map. Ensemble Kalman methods and Langevin-based methods (the latter use the derivative of the parameter-to-data map) are compared in this light. The ensemble Kalman methods are shown to behave favorably in the presence of noise in the parameter-to-data map, whereas Langevin methods are adversely affected. On the other hand, Langevin methods have the correct equilibrium distribution in the setting of noise-free forward models, while ensemble Kalman methods only provide an uncontrolled approximation, except in the linear case. Therefore a new class of algorithms, ensemble Gaussian process samplers, which combine the benefits of both ensemble Kalman and Langevin methods, are introduced and shown to perform favorably.

Item Type:Article
Related URLs:
URLURL TypeDescription Paper
Dunbar, Oliver R. A.0000-0001-7374-0382
Stuart, Andrew M.0000-0001-9091-7266
Wolfram, Marie-Therese0000-0003-1133-8253
Additional Information:© 2022 Society for Industrial and Applied Mathematics. Received by the editors April 8, 2021; accepted for publication (in revised form) by G. Gottwald March 8, 2022; published electronically June 21, 2022. The work of the second author was supported by the UKRI Strategic Priorities Fund under EPSRC Grant EP/T001569/1, particularly the "Digital Twins for Complex Engineering Systems" theme within that grant, and by the Alan Turing Institute. The work of the fourth author was supported by New Frontier Grant NST-0001 of the Austrian Academy of Sciences. The work of the third author was supported by the NSF through grants AGS-1835860 and DMS-1818977 and by the Office of Naval Research (award N00014-17-1-2079). The work of the third and fourth authors was supported by a Royal Society International Exchange Grant.
Funding AgencyGrant Number
Engineering and Physical Sciences Research Council (EPSRC)EP/T001569/1
Alan Turing InstituteUNSPECIFIED
Österreichische Akademie der WissenschaftenNST-0001
Office of Naval Research (ONR)N00014-17-1-2079
Subject Keywords:ensemble methods, ensemble Kalman sampler, Langevin sampling, Gaussian process regression, multiscale analysis
Issue or Number:2
Classification Code:AMS subject classifications: 60H30, 35B27, 60G15, 82C80, 65C35, 62F15
Record Number:CaltechAUTHORS:20210412-121307581
Persistent URL:
Official Citation:Ensemble Inference Methods for Models With Noisy and Expensive Likelihoods. Oliver R. A. Dunbar, Andrew B. Duncan, Andrew M. Stuart, and Marie-Therese Wolfram. SIAM Journal on Applied Dynamical Systems 2022 21:2, 1539-1572; DOI: 10.1137/21M1410853
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:108700
Deposited By: Tony Diaz
Deposited On:13 Apr 2021 21:34
Last Modified:12 Aug 2022 19:45

Repository Staff Only: item control page