A Caltech Library Service

Scientific multi-agent reinforcement learning for wall-models of turbulent flows

Bae, H. Jane and Koumoutsakos, Petros (2022) Scientific multi-agent reinforcement learning for wall-models of turbulent flows. Nature Communications, 13 . Art. No. 1443. ISSN 2041-1723. PMCID PMC8931082. doi:10.1038/s41467-022-28957-7.

[img] PDF - Published Version
Creative Commons Attribution.

[img] PDF - Submitted Version
Creative Commons Attribution.

[img] PDF (Peer Review File) - Supplemental Material
Creative Commons Attribution.


Use this Persistent URL to link to this item:


The predictive capabilities of turbulent flow simulations, critical for aerodynamic design and weather prediction, hinge on the choice of turbulence models. The abundance of data from experiments and simulations and the advent of machine learning have provided a boost to turbulence modeling efforts. However, simulations of turbulent flows remain hindered by the inability of heuristics and supervised learning to model the near-wall dynamics. We address this challenge by introducing scientific multi-agent reinforcement learning (SciMARL) for the discovery of wall models for large-eddy simulations (LES). In SciMARL, discretization points act also as cooperating agents that learn to supply the LES closure model. The agents self-learn using limited data and generalize to extreme Reynolds numbers and previously unseen geometries. The present simulations reduce by several orders of magnitude the computational cost over fully-resolved simulations while reproducing key flow quantities. We believe that SciMARL creates unprecedented capabilities for the simulation of turbulent flows.

Item Type:Article
Related URLs:
URLURL TypeDescription ItemData ItemCode CentralArticle Paper
Bae, H. Jane0000-0001-6789-6209
Koumoutsakos, Petros0000-0001-8337-2122
Additional Information:© The Author(s) 2022. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit Received 29 May 2021; Accepted 14 February 2022; Published 17 March 2022. The authors acknowledge the support of Air Force Office of Scientific Research (AFOSR) Multidisciplinary University Research Initiative (MURI) project: Prediction, Statistical Quantification, and Mitigation of Extreme Events Caused by Exogenous Causes or Intrinsic Instabilities under grant number FA9550-21-1-0058. Computational resources were provided by the Swiss National Supercomputing Centre (CSCS) Project s929. Data availability: All the data analyzed in this paper were produced with an in-house flow solver and an open-source reinforcement learning software described in the code availability statement. Reference data and the scripts used to produce the data figures is available through GitHub ( Code availability: The wall-modeled large-eddy simulations were performed with a in-house flow solver, which is available on demand. The wall models were trained with the reinforcement learning library smarties ( Contributions: H.J.B. jointly conceived the study with P.K., designed and performed experiments, analyzed the data, and wrote the paper; P.K. devised the concept of SciMARL, supervised the project, and edited the manuscript. The authors declare no competing interests. Peer review information: Nature Communications thanks the anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)FA9550-21-1-0058
Subject Keywords:Computational science; Engineering; Mathematics and computing
PubMed Central ID:PMC8931082
Record Number:CaltechAUTHORS:20220317-189733300
Persistent URL:
Official Citation:Bae, H.J., Koumoutsakos, P. Scientific multi-agent reinforcement learning for wall-models of turbulent flows. Nat Commun 13, 1443 (2022).
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:113956
Deposited By: Tony Diaz
Deposited On:17 Mar 2022 23:10
Last Modified:04 Apr 2022 16:50

Repository Staff Only: item control page