CaltechAUTHORS
  A Caltech Library Service

Sparse Cholesky Factorization by Kullback-Leibler Minimization

Schäfer, Florian and Katzfuss, Matthias and Owhadi, Houman (2021) Sparse Cholesky Factorization by Kullback-Leibler Minimization. SIAM Journal on Scientific Computing, 43 (3). A2019-A2046. ISSN 1064-8275. doi:10.1137/20M1336254. https://resolver.caltech.edu/CaltechAUTHORS:20201109-155534680

[img] PDF - Published Version
See Usage Policy.

10MB
[img] PDF - Submitted Version
See Usage Policy.

8MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20201109-155534680

Abstract

We propose to compute a sparse approximate inverse Cholesky factor L of a dense covariance matrix Θ by minimizing the Kullback--Leibler divergence between the Gaussian distributions N(0,Θ) and N(0,L^(−⊤)L⁻¹), subject to a sparsity constraint. Surprisingly, this problem has a closed-form solution that can be computed efficiently, recovering the popular Vecchia approximation in spatial statistics. Based on recent results on the approximate sparsity of inverse Cholesky factors of Θ obtained from pairwise evaluation of Green's functions of elliptic boundary-value problems at points {x_i}_(1≤i≤N) ⊂ ℝ^d, we propose an elimination ordering and sparsity pattern that allows us to compute ϵ-approximate inverse Cholesky factors of such Θ in computational complexity O(N log(N/ϵ)^d) in space and O(N log(N/ϵ)^(2d)) in time. To the best of our knowledge, this is the best asymptotic complexity for this class of problems. Furthermore, our method is embarrassingly parallel, automatically exploits low-dimensional structure in the data, and can perform Gaussian-process regression in linear (in N) space complexity. Motivated by its optimality properties, we propose applying our method to the joint covariance of training and prediction points in Gaussian-process regression, greatly improving stability and computational cost. Finally, we show how to apply our method to the important setting of Gaussian processes with additive noise, compromising neither accuracy nor computational complexity.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1137/20M1336254DOIArticle
https://arxiv.org/abs/2004.14455arXivDiscussion Paper
ORCID:
AuthorORCID
Katzfuss, Matthias0000-0001-7496-7992
Owhadi, Houman0000-0002-5677-1600
Additional Information:© 2021, Society for Industrial and Applied Mathematics. Submitted to the journal's Methods and Algorithms for Scientific Computing section May 6, 2020; accepted for publication (in revised form) January 27, 2021; published electronically June 3, 2021. Funding: The work of the first and third authors was supported by the Air Force Office of Scientific Research under award FA9550-18-1-0271 (Games for Computation and Learning) and by the Office of Naval Research under award N00014-18-1-2363. The work of the second author was partially supported by National Science Foundation (NSF) through grants DMS-1654083, DMS-1953005, and CCF-1934904. We thank the two anonymous referees for their constructive feedback, which helped us to improve the article.
Funders:
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)FA9550-18-1-0271
Office of Naval Research (ONR)N00014-18-1-2363
NSFDMS-1654083
NSFDMS-1953005
NSFCCF-1934904
Subject Keywords:Cholesky factorization, screening effect, Vecchia approximation, factorized approximate inverse, Gaussian process regression, integral equation
Issue or Number:3
Classification Code:AMS subject classifications: 5F30, 42C40, 65F50, 65N55, 65N75, 60G42, 68W40
DOI:10.1137/20M1336254
Record Number:CaltechAUTHORS:20201109-155534680
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20201109-155534680
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:106572
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:10 Nov 2020 15:25
Last Modified:20 Aug 2021 16:56

Repository Staff Only: item control page