A Caltech Library Service

Score-based Diffusion Models in Function Space

Lim, Jae Hyun and Kovachki, Nikola B. and Baptista, Ricardo and Beckham, Christopher and Azizzadenesheli, Kamyar and Kossaifi, Jean and Voleti, Vikram and Song, Jiaming and Kreis, Karsten and Kautz, Jan and Pal, Christopher and Vahdat, Arash and Anandkumar, Anima (2023) Score-based Diffusion Models in Function Space. . (Unpublished)

[img] PDF - Submitted Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Diffusion models have recently emerged as a powerful framework for generative modeling. They consist of a forward process that perturbs input data with Gaussian white noise and a reverse process that learns a score function to generate samples by denoising. Despite their tremendous success, they are mostly formulated on finite-dimensional spaces, e.g. Euclidean, limiting their applications to many domains where the data has a functional form such as in scientific computing and 3D geometric data analysis. In this work, we introduce a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space. In DDOs, the forward process perturbs input functions gradually using a Gaussian process. The generative process is formulated by integrating a function-valued Langevin dynamic. Our approach requires an appropriate notion of the score for the perturbed data distribution, which we obtain by generalizing denoising score matching to function spaces that can be infinite-dimensional. We show that the corresponding discretized algorithm generates accurate samples at a fixed cost that is independent of the data resolution. We theoretically and numerically verify the applicability of our approach on a set of problems, including generating solutions to the Navier-Stokes equation viewed as the push-forward distribution of forcings from a Gaussian Random Field (GRF).

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Kovachki, Nikola B.0000-0002-3650-2972
Beckham, Christopher0000-0001-6337-4526
Azizzadenesheli, Kamyar0000-0001-8507-1868
Kossaifi, Jean0000-0002-4445-3429
Voleti, Vikram0000-0003-0941-7227
Song, Jiaming0000-0003-2794-2180
Kautz, Jan0000-0002-8830-429X
Pal, Christopher0000-0001-6534-2114
Anandkumar, Anima0000-0002-6974-6797
Additional Information:Attribution 4.0 International (CC BY 4.0)
Record Number:CaltechAUTHORS:20230316-153712038
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:120082
Deposited By: George Porter
Deposited On:16 Mar 2023 22:37
Last Modified:16 Mar 2023 22:37

Repository Staff Only: item control page