arXiv:2301.08290v1 [physics.flu-dyn] 19 Jan 2023
arXiv preprint
1
Forecasting subcritical cylinder wakes with Fourier
Neural Operators
Peter I Renn
1
†
, Cong Wang
1
, Sahin Lale
1
, Zongyi Li
1
, Anima Anandkumar
1
,
and Morteza Gharib
1
1
Division of Engineering and Applied Science, California In
stitute of Technology, Pasadena, CA 91125,
USA
We apply Fourier neural operators (FNOs), a state-of-the-a
rt operator learning technique,
to forecast the temporal evolution of experimentally measu
red velocity fields. FNOs are a
recently developed machine learning method capable of appr
oximating solution operators
to systems of partial differential equations through data al
one. The learned FNO solution
operator can be evaluated in milliseconds, potentially ena
bling faster-than-real-time modeling
for predictive flow control in physical systems. Here we use F
NOs to predict how physical
fluid flows evolve in time, training with particle image veloc
imetry measurements depicting
cylinder wakes in the subcritical vortex shedding regime. W
e train separate FNOs at Reynolds
numbers ranging from
Re
=
240 to
Re
=
3060 and study how increasingly turbulent flow
phenomena impact prediction accuracy. We focus here on a sho
rt prediction horizon of ten
non-dimensionalizedtime-steps, as would be relevant for p
roblems of predictive flow control.
We find that FNOs are capable of accurately predicting the evo
lution of experimental velocity
fields throughout the range of Reynolds numbers tested (L2 no
rm error
<
0
.
1) despite being
provided with limited and imperfect flow observations. Give
n these results, we conclude
that this method holds significant potential for real-time p
redictive flow control of physical
systems.
1. Introduction
Vortex shedding in a cylinder wake is among the most fundamen
tal and well-studied problems
in fluid mechanics. This phenomenon, otherwise known as a Kár
mán vortex street, is still
relevant for a huge array of industries and applications tod
ay. The vortex shedding process has
been studied in detail for decades with several comprehensi
ve review papers on the subject,
so we will keep our descriptions here brief (
Williamson 1996
b
). Vortex shedding is first
observed around
Re
=
50, at which point it has been observed that an instability oc
curs in
the previously steady recirculation regions of the wake. Th
is instability results in a famously
beautiful and well-ordered pattern of laminar alternating
vortices convecting downstream
away from the cylinder. There has been some experimental var
iation observed in defining
where the laminar vortex shedding regime ends but it is gener
ally placed around
Re
= 190, at
which point small-scale three-dimensional instabilities
form and the transition to turbulence
begins (
Williamson 1996
a
). Following transition, the behavior of cylinder wakes fro
m
Re
=
300 to
Re
= 200 000 was labeled the "irregular range" by
Roshko
(
1958
) (otherwise known
†
Email address for correspondence: prenn@caltech.edu
2
as the sub-critical regime). This regime is characterized b
y increasing three-dimensional
effects, irregular velocity fluctuations, and the transitio
n of the outer shear layer (
Re
= 1200)
(
Williamson 1996
b
).
Modeling vortical wakes in the “irregular range” is possibl
e in standard computationalfluid
dynamics (CFD) simulations, however, can be computational
ly expensive and time intensive
(
Pereira
et al.
2017
). As an alternative to the conventional numerical solvers w
idely adopted
by the CFD community, there has been growing interest in appl
ications of data-driven
methods for modeling the time evolution of fluid flows. In cont
rast to standard CFD solvers,
data-driven methods require data and time up-front but are s
ignificantly faster to evaluate
once trained.
Previously proposed data-driven methods for predicting th
e time evolution of fluid flows
primarily focus on neural networks with varying structures
(e.g. convolutional layers, decon-
volutional layers, long short-term memory cells, and varia
tional autoencoders, etc.). Several
of these works involve applying neural networks to predict t
he evolution of computational
simulations of laminar flow around a cylinder (
Hasegawa
et al.
2020
b
,
a
;
Moriomoto
et al.
2021
).
Srinivasan
et al.
(
2019
) and
Nakamura
et al.
(
2021
) apply similar neural network
approaches to computational simulations of turbulent shea
r layers and three-dimensional
channel flows, respectively. In both cases, the data-driven
models match statistical quantities
of the flow well but struggle to make accurate instantaneous p
redictions.
Han
et al.
(
2019
)
demonstrate a method for predicting the time evolution of cy
linder flows from computational
simulations at various Reynolds numbers including some tur
bulent, but require dozens
of previous time-steps as input.
Wu
et al.
(
2022
) propose a multi-resolution convolutional
interaction network to make temporal predictions for cylin
der flow data from eddy-viscosity
turbulence models in the moderate subcritical regime, but t
he model is outperformed by
basic linear dynamic mode decomposition for a cylinder flow w
ith constant inlet velocity.
Fukami
et al.
(
2021
b
) combine a convolutional neural network auto-encoder with
sparse
identification of nonlinear dynamics (i.e. SINDy) to model l
aminar cylinder flow data
generated by direct numerical simulation, as well as a shear
flow model. This method
is capable of learning the latent dynamics of both cases, how
ever, the resulting models are
sensitive to noisy observations and require problem-speci
fic handling of learning parameters.
These previous approaches are all limited to predicting num
erical simulations of fluid
flows, and very few of them include considerations of the impa
ct of noisy or imperfect
measurements. Additionally, approximating with standard
neural networks means that the
learned dynamics can only be evaluated at fixed points used du
ring training.
Neural operators are a recently developed class of machine l
earning techniques that are
particularly well-poised for applications within fluid mec
hanics. Neural operators share much
of the basic structure of standard neural networks commonly
used to approximate functions
but are distinct in their ability to approximate operators.
Directly approximating operators
allow for these methods to learn mappings between infinite-d
imensional function spaces
(i.e. sets of functions), whereas standard neural networks
are typically used to approximate
a single function. Learning mappings between infinite-dime
nsional function spaces enables
the approximation of solution operators to systems of parti
al differential equations (PDEs)
such as the constitutive equations underlying physical pro
cesses studied in fluid mechanics.
Introduced by
Li
et al.
(
2021
), Fourier neural operators (FNOs) combine linear transfor
ms
in Fourier space with local non-linear activation function
s as found in standard neural
networks. The linear transforms performed in Fourier space
(via the Fast Fourier Transform)
serve as global integral operators in real space, which allo
w FNOs to learn non-local effects
for highly non-linear operators. Because they directly app
roximate solution operators, FNOs
are also discretization invariant meaning that the learned
operator can be evaluated on an
arbitrary mesh regardless of the discretization of trainin
g data.
Li
et al.
(
2021
) previously
3
applied FNOs to computational solutions of the Navier-Stok
es equation, achieving zero-shot
super-resolution and successfully predicting the tempora
l development of solutions to the
vorticity equation. FNOs were also demonstrated on fluid flow
s in a follow-up work by
Li
et al.
(
2022
), where they approximated solutions to a wavy pipe flow and a t
ransonic
airfoil to demonstrate the approach on general geometries.
Other neural operator variations
(e.g. DeepONet (
Lu
et al.
2021
)) have been applied to computational flow data as well in
different contexts (
Di Leoni
et al.
2022
;
Lin
et al.
2021
).
Orders of magnitude faster than numerical solvers (
Kovachki
et al.
2022
), FNOs can be
evaluated in just milliseconds which gives them the potenti
al to predict the evolution of fluid
flows faster than they occurin real-time. Accurate, full-fie
ld predictions could have significant
implications for real-world engineering problems such as t
he mitigation of atmospheric gusts
and control of turbulent boundary layers. However previous
studies using neural operators
on fluid mechanics have focused on problems in computational
simulation with perfect
information, convenient parameters, and known boundary co
nditions. To the best of our
knowledge, this is the first application of operator learnin
g on experimental measurements
of fluid flow.
Here we explore the potential for FNOs as a real-time-capabl
e machine learning technique
for the prediction of fluid flows. We train FNOs on experimenta
l flow data measured via
particle image velocimetry (PIV) to forecast the time evolu
tion of cylinder wakes at a range
of Reynolds numbers in the subcritical vortex shedding regi
me. The FNO learns to predict
the evolution of the flow over ten time-steps, which are non-d
imensionalized so that the
resulting prediction horizon is equivalent to roughly 1.7 d
iameters of translation in the free
stream. This relatively short forecast window is chosen to d
emonstrate FNOs for modeling
flow phenomena in the context of predictive control methods w
ith finite planning horizons.
We find that this operator learning approach accurately pred
icts instantaneous velocity fields
over the full range of Reynolds numbers tested (L2 norm error
<
0
.
1) and is robust to
experimental noise.
2. Experimental setup
2.1.
Data acquisition
Our data was collected via experiments performed in a small f
ree-surface water tunnel with
a test section of 0.15 m (W)
×
0.15 m (H) and a length of 0.61 m. We performed tests at flow
speeds ranging from
*
= 0.02 m s
−
1
to
*
= 0.40 m s
−
1
. The corresponding Reynolds numbers
range from about
Re
= 240 to
Re
= 3100, where we define Reynolds number as
Re
=
*
/
a
with
being the cylinder diameter and
a
being the kinematic viscosity. The cylinder diameter
is held constant at
= 9.53
×
10
−
3
m and is fully submerged and fixed to the tunnel walls
on both sides. The cylinder is made of cast acrylic and is moun
ted approximately equidistant
from the free surface and the tunnel floor to minimize the impa
ct of either boundary on the
vortex wake.
Figure
1
depicts the region of interest relative to the cylinder and t
he tunnel boundaries.
Here
!
1
= 0.125 m (
≈
13
) and
!
2
= 0.084 m (
≈
9
). This region, located immediately
behind the cylinder, is illuminated by a laser sheet at a stre
amwise cross section near the
center of the tunnel. A high-speed camera (IDT XSM-3520 set t
o 2144
×
1440 resolution)
is used to record the flow in this region. The frame rate of the c
amera is adjusted based
on the mean flow speed to maintain a near-constant non-dimens
ional time between frames
regardless of tunnel speed. The non-dimensional time, othe
rwise known as the formation
time, for a cylinder flow is given by
Jeon & Gharib
(
2004
) as:
4
U
D
L
1
L
2
Figure 1: Schematic of the imaging region of interest, outli
ned by the black dashed line,
relative to the water tunnel and cylinder.
C
∗
=
*C
(2.1)
Here
C
∗
is the non-dimensional time and
C
is the dimensional time. We used a two-
dimensional two-component particle image velocimetry (2D
2C-PIV) algorithm to calculate
velocity fields by correlating consecutive image pairs. We s
et an interrogation window size
of 32
×
32 pixels with 50% overlap, giving a spatial resolution of
<
0
.
1
for our resulting
velocity fields. Although our measurements only resolve vel
ocity components parallel to the
laser sheet, flow in the subcritical vortex shedding regime i
s known to be three-dimensional
(
Williamson 1996
b
). We also observed additional three-dimensional effects in
the form of an
oblique vortex shedding angle, which is known to have implic
ations on the
(C
−
'4
relationship
(
Hammache & Gharib 1991
;
Prasad & Williamson 1997
b
). In addition to the additional
challenges associated with increased measurement noise fr
om out-of-plane velocities and a
more complicated learning problem (learning the two-dimen
sional evolution of an inherently
three-dimensional flow), this means that matching the non-d
imensional time (
C
∗
) defined in
equation
2.1
does not guarantee equivalent Strouhal periods.
2.2.
Fourier neural operators
Neural operators are distinct from standard neural network
s in their unique ability to directly
approximate operators, such as mappings between infinite-d
imensional function spaces, from
data alone. This includes operators that map between sets of
functions related by a family of
PDEs (e.g. the Navier-Stokes equations), making these meth
ods particularly well-suited for
problems in fluid mechanics. . Additionally, the learned sol
ution operator can be evaluated
in just a few milliseconds using a standard GPU, making it ord
ers of magnitude faster than
the pseudo-spectral method
Kovachki
et al.
(
2022
). Introduced by
Li
et al.
(
2021
), Fourier
neural operators (FNOs) are a powerful form of neural operat
or that are guaranteed universal
approximation for continuous operators (
Kovachki
et al.
2021
,
2022
). They are also mesh-
invariant, meaning that they can be trained and evaluated wi
th data of varying resolutions.
The architecture of an FNO, shown in figure
2
, is made up of unique Fourier layers. These
Fourier layers consist of paths: non-linear activation fun
ctions as found in classical neural
networks, and a linear transform of the input signal perform
ed in Fourier space. The non-
linear activation function path serves to approximate loca
l non-linearities, and the transform
in Fourier space serves as a global integral operator to acco
unt for non-local effects in real
5
Figure 2: Diagram showing composition of Fourier layer in FN
O. Taken, with permission,
from (
Li
et al.
2021
).
space. The paths are then combined and passed forward. The we
ights in each layer are trained
through back-propagation to minimize the selected loss fun
ction, in a way that parallels the
training of standard neural networks. Details on the mathem
atical principles underlying FNOs
and neural operators more generally can be found in
Li
et al.
(
2021
),
Kovachki
et al.
(
2022
),
and
Kovachki
et al.
(
2021
). The FNO hyperparameters used in this work can be found in
appendix
A
.
3. Results
3.1.
Prediction approach
In this study, we use FNOs to forecast the time evolution of bo
th
G
and
H
components of
velocity fields (
D, E
) in the wake of a cylinder at various Reynolds numbers. In thi
s context,
FNOs can learn to predict future states of the velocity fields
based on current observations.
While not necessary for learning with FNOs, we preprocessed
the data by subtracting out
the mean field. This approach, borrowing from stability theo
ry (
Landau & Lifshitz 1987
)[pp.
95], essentially decomposed the velocity into steady and un
steady parts:
풖
(
풙
, C
)
=
풖
0
(
풙
) +
풖
′
(
풙
, C
)
(3.1)
We then can substitute this into the Navier-Stokes equation
s. Assuming the steady part alone
satisfies the time-independent equations and omitting
풖
′
(
풙
, C
)
terms of order greater than
one, we are left with
m
풖
′
mC
+ (
풖
0
· ∇)
풖
′
+ (
풖
′
· ∇)
풖
0
=
∇
?
′
d
+
a
Δ
풖
′
,
∇ ·
풖
′
=
0
(3.2)
where
?
′
is the unsteady pressure component (i.e.
?
(
풙
, C
)
=
?
0
(
풙
) +
?
′
(
풙
, C
)
). Therefore, we
are left with a set of homogeneous linear differential equati
ons (
Landau & Lifshitz 1987
),
which are perhaps more readily learned by the FNO.
In addition to simplifying the underlying differential equa
tions, subtracting out the time-
averaged velocity ensures that unsteady fluctuations are no
t dominated by the large free-
stream velocity bias. This is common to many modal analysis t
echniques used in studying
fluid flows (e.g. proper orthogonal decomposition) (
Taira
et al.
2017
) and likely has similar
benefits in frequency-based learning methods like FNOs. A co
mparison of performance
between FNOs provided the full velocity components (
풖
(
풙
, C
)
) and fluctuation velocity
components (
풖
′
(
풙
, C
)
) can be found in the appendix. Since we train the FNOs to predi
ct
the fluctuations alone, we can then reconstruct the full flow fi
eld by summing the mean and
fluctuating velocity components as defined in equation
3.1
.
While FNOs can make predictions from only one time-step as in
put, we chose to use two
time-steps to reduce the effects of experimental errors. Fro
m this two time-step input, the
6
u
v
t
= 1
u
v
u*
v*
t
= 2
FNO
t
= 3
u*
v*
u*
v*
t
= 3
u
v
t
= 2
t
= 4
u*
v*
u*
v*
u*
v*
t
=
n
- 2
t
=
n
- 1
t
=
n
FNO
FNO
Figure 3: Recursive application of Fourier Neural Operator
s (FNOs).
model predicts the state of the flow field at the following time
-step. During training, we set
a desired number of time-steps to forecast. The model is then
applied recursively, as shown
in figure
3
, to reach the desired number of steps. The loss is calculated
at each recursive step
and summed to find the overall loss in the prediction. The FNO t
ries to minimize this loss
through the back-propagation algorithm, similar to a class
ic neural network. Because they
are applied recursively, the FNO models can also be used to pr
edict time-steps beyond the
trained horizon.
In analyzing predictions, we define the error of the flow field,
n
, at a given time-step
C
as
calculated as the L2 error norm:
n
(
C
)
=
k
풒
∗
푡
−
풒
푡
k
2
k
풒
푡
k
2
(3.3)
where
풒
푡
is a
#
×
1 vector containing the two-component velocity field, such t
hat
풒
푡
=
D
(
G
1
, H
1
, C
)
D
(
G
1
, H
2
, C
)
...
D
(
G
1
, H
푚
, C
)
D
(
G
2
, H
1
, C
)
...
D
(
G
푛
, H
푚
, C
)
E
(
G
1
, H
1
, C
)
...
E
(
G
푛
, H
푚
, C
)
(3.4)
and
풒
∗
푡
is the FNO estimate of
풒
푡
. The quantities
D
(
G
푖
, H
푗
, C
)
and
E
(
G
푖
, H
푗
, C
)
in equation
3.4
are the full, reconstructed velocity components at
(
G
푖
, H
푗
)
and time
C
.
We non-dimensionalized the time-step across Reynolds numb
ers using the same formula-
tion shown in equation
2.1
. This time-step was set to be exactly five times the inter-fra
me time
for each Reynolds number (
Δ
C
≈
0
.
17). This relatively large time-step was chosen because
the measured difference in consecutive velocity fields was on
the order which we would