CaltechAUTHORS
  A Caltech Library Service

I²SB: Image-to-Image Schrödinger Bridge

Liu, Guan-Horng and Vahdat, Arash and Huang, De-An and Theodorou, Evangelos A. and Nie, Weili and Anandkumar, Anima (2023) I²SB: Image-to-Image Schrödinger Bridge. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20230316-153732528

[img] PDF - Submitted Version
See Usage Policy.

25MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20230316-153732528

Abstract

We propose Image-to-Image Schrödinger Bridge (I²SB), a new class of conditional diffusion models that directly learn the nonlinear diffusion processes between two given distributions. These diffusion bridges are particularly useful for image restoration, as the degraded images are structurally informative priors for reconstructing the clean images. I²SB belongs to a tractable class of Schrödinger bridge, the nonlinear extension to score-based models, whose marginal distributions can be computed analytically given boundary pairs. This results in a simulation-free framework for nonlinear diffusions, where the I²SB training becomes scalable by adopting practical techniques used in standard diffusion models. We validate I²SB in solving various image restoration tasks, including inpainting, super-resolution, deblurring, and JPEG restoration on ImageNet 256x256 and show that I²SB surpasses standard conditional diffusion models with more interpretable generative processes. Moreover, I²SB matches the performance of inverse methods that additionally require the knowledge of the corruption operators. Our work opens up new algorithmic opportunities for developing efficient nonlinear diffusion models on a large scale. scale. Project page: https://i2sb.github.io/


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2302.05872arXivDiscussion Paper
https://i2sb.github.io/Related ItemProject website
ORCID:
AuthorORCID
Liu, Guan-Horng0000-0002-8989-7568
Huang, De-An0000-0002-6945-7768
Theodorou, Evangelos A.0000-0002-0834-5738
Anandkumar, Anima0000-0002-6974-6797
Record Number:CaltechAUTHORS:20230316-153732528
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20230316-153732528
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:120085
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:16 Mar 2023 22:48
Last Modified:16 Mar 2023 22:48

Repository Staff Only: item control page