Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published 2015 | Published + Submitted
Journal Article Open

Algorithms for Kullback--Leibler Approximation of Probability Measures in Infinite Dimensions


In this paper we study algorithms to find a Gaussian approximation to a target measure defined on a Hilbert space of functions; the target measure itself is defined via its density with respect to a reference Gaussian measure. We employ the Kullback--Leibler divergence as a distance and find the best Gaussian approximation by minimizing this distance. It then follows that the approximate Gaussian must be equivalent to the Gaussian reference measure, defining a natural function space setting for the underlying calculus of variations problem. We introduce a computational algorithm which is well-adapted to the required minimization, seeking to find the mean as a function, and parameterizing the covariance in two different ways: through low rank perturbations of the reference covariance and through Schrödinger potential perturbations of the inverse reference covariance. Two applications are shown: to a nonlinear inverse problem in elliptic PDEs and to a conditioned diffusion process. These Gaussian approximations also serve to provide a preconditioned proposal distribution for improved preconditioned Crank--Nicolson Monte Carlo--Markov chain sampling of the target distribution. This approach is not only well-adapted to the high dimensional setting, but also behaves well with respect to small observational noise (resp., small temperatures) in the inverse problem (resp., conditioned diffusion).

Additional Information

© 2015, Society for Industrial and Applied Mathematics. Submitted to the journal's Methods and Algorithms for Scientific Computing section August 11, 2014; accepted for publication (in revised form) April 9, 2015; published electronically November 17, 2015. [Simpson's] work was supported in part by DOE Award DE-SC0002085 and NSF PIRE Grant OISE-0967140. The first author's work was supported by EPSRC, ERC, and ONR. The second author's work was supported by an EPSRC First Grant.

Attached Files

Submitted - 1408.1920.pdf

Published - stuart120.pdf


Files (6.1 MB)
Name Size Download all
4.1 MB Preview Download
2.0 MB Preview Download

Additional details

August 20, 2023
August 20, 2023