A Caltech Library Service

Incremental Fourier Neural Operator

Zhao, Jiawei and George, Robert Joseph and Zhang, Yifei and Li, Zongyi and Anandkumar, Anima (2022) Incremental Fourier Neural Operator. . (Unpublished)

[img] PDF - Accepted Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Recently, neural networks have proven their impressive ability to solve partial differential equations (PDEs). Among them, Fourier neural operator (FNO) has shown success in learning solution operators for highly non-linear problems such as turbulence flow. FNO is discretization-invariant, where it can be trained on low-resolution data and generalizes to problems with high-resolution. This property is related to the low-pass filters in FNO, where only a limited number of frequency modes are selected to propagate information. However, it is still a challenge to select an appropriate number of frequency modes and training resolution for different PDEs. Too few frequency modes and low-resolution data hurt generalization, while too many frequency modes and high-resolution data are computationally expensive and lead to over-fitting. To this end, we propose Incremental Fourier Neural Operator (IFNO), which augments both the frequency modes and data resolution incrementally during training. We show that IFNO achieves better generalization (around 15% reduction on testing L2 loss) while reducing the computational cost by 35%, compared to the standard FNO. In addition, we observe that IFNO follows the behavior of implicit regularization in FNO, which explains its excellent generalization ability.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
George, Robert Joseph0000-0001-8201-1616
Li, Zongyi0000-0003-2081-9665
Anandkumar, Anima0000-0002-6974-6797
Additional Information:Attribution 4.0 International (CC BY 4.0). We thank NVIDIA for computational support. Z. Li gratefully acknowledges the financial support from the PIMCO Fellows and Amazon AI4Science Fellows programs. A. Anandkumar is supported in part by Bren endowed chair.
Funding AgencyGrant Number
Amazon AI4Science FellowshipUNSPECIFIED
Bren Professor of Computing and Mathematical SciencesUNSPECIFIED
Record Number:CaltechAUTHORS:20221221-004746129
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:118563
Deposited By: George Porter
Deposited On:21 Dec 2022 20:45
Last Modified:21 Dec 2022 20:45

Repository Staff Only: item control page