Published March 9, 2024 | Submitted v3
Discussion Paper Open

Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes

  • 1. ROR icon Courant Institute of Mathematical Sciences
  • 2. ROR icon California Institute of Technology
  • 3. ROR icon Georgia Institute of Technology

Abstract

In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its \textit{partial derivatives}, a result of the PDE constraint and for which fast algorithms are scarce.

The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute ϵ-approximate inverse Cholesky factors of the kernel matrices with complexity O(Nlogd(N/ϵ)) in space and O(Nlog2d(N/ϵ)) in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm's near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Ampère equations.

Acknowledgement

YC and HO acknowledge support from the Air Force Office of Scientific Research under MURI award number FA9550-20-1-0358 (Machine Learning and PhysicsBased Modeling and Simulation). YC is also partly supported by NSF Grants DMS2205590. HO also acknowledges support from the Department of Energy under award number DE-SC0023163 (SEA-CROGS: Scalable, Efficient and Accelerated Causal Reasoning Operators, Graphs and Spikes for Earth and Embedded Systems). FS acknowledges support from the Office of Naval Research under award number N00014-23-1-2545 (Untangling Computation). We thank Xianjin Yang for helpful comments on an earlier version of this article.

Copyright and License

License: arXiv.org perpetual non-exclusive license

Files

2304.01294v3.pdf
Files (1.2 MB)
Name Size Download all
md5:cd6b7be7d03df6d85aa38e1e3e1f31fe
1.2 MB Preview Download

Additional details

Created:
November 15, 2024
Modified:
November 15, 2024