A Caltech Library Service

Learning efficient navigation in vortical flow fields

Gunnarson, Peter and Mandralis, Ioannis and Novati, Guido and Koumoutsakos, Petros and Dabiri, John O. (2021) Learning efficient navigation in vortical flow fields. Nature Communications, 12 . Art. No. 7143. ISSN 2041-1723. PMCID PMC8654940. doi:10.1038/s41467-021-27015-y.

[img] PDF - Published Version
Creative Commons Attribution.

PDF - Submitted Version
See Usage Policy.

[img] PDF - Supplemental Material
Creative Commons Attribution.

[img] PDF (Description of additional Supplementary File) - Supplemental Material
Creative Commons Attribution.

[img] Video (MPEG) (Supplementary Video 1) - Supplemental Material
Creative Commons Attribution.

[img] Video (MPEG) (Supplementary Video 2) - Supplemental Material
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Efficient point-to-point navigation in the presence of a background flow field is important for robotic applications such as ocean surveying. In such applications, robots may only have knowledge of their immediate surroundings or be faced with time-varying currents, which limits the use of optimal control techniques. Here, we apply a recently introduced Reinforcement Learning algorithm to discover time-efficient navigation policies to steer a fixed-speed swimmer through unsteady two-dimensional flow fields. The algorithm entails inputting environmental cues into a deep neural network that determines the swimmer’s actions, and deploying Remember and Forget Experience Replay. We find that the resulting swimmers successfully exploit the background flow to reach the target, but that this success depends on the sensed environmental cue. Surprisingly, a velocity sensing approach significantly outperformed a bio-mimetic vorticity sensing approach, and achieved a near 100% success rate in reaching the target locations while approaching the time-efficiency of optimal navigation trajectories.

Item Type:Article
Related URLs:
URLURL TypeDescription CentralArticle Paper ItemCode
Mandralis, Ioannis0000-0001-5270-0672
Novati, Guido0000-0003-0681-4892
Koumoutsakos, Petros0000-0001-8337-2122
Dabiri, John O.0000-0002-6722-9008
Additional Information:© The Author(s) 2021. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. Received 04 March 2021. Accepted 01 November 2021. Published 08 December 2021. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 1745301. P.G. was supported by this fellowship. Data availability. All data generated and discussed in this study are available within the article and its supplementary files, or are available from the authors upon request. Code availability. The Deep Reinforcement Learning algorithm V-RACER is available at Contributions. P.G., I.M., G.N., P.K., and J.O.D. designed research and were involved in discussions to interpret the results; P.G. performed research and analyzed results; G.N. and P.K. developed the V-RACER algorithm; G.N. wrote the software implementation of V-RACER; I.M. simulated the cylinder flow field; P.G. drafted the paper, and all authors helped edit and review. The authors declare no competing interests.
Funding AgencyGrant Number
NSF Graduate Research FellowshipDGE‐1745301
Subject Keywords:Biomechanics; Computational science; Marine biology; Mechanical engineering
PubMed Central ID:PMC8654940
Record Number:CaltechAUTHORS:20210322-104504056
Persistent URL:
Official Citation:Gunnarson, P., Mandralis, I., Novati, G. et al. Learning efficient navigation in vortical flow fields. Nat Commun 12, 7143 (2021).
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:108507
Deposited By: Tony Diaz
Deposited On:23 Mar 2021 15:57
Last Modified:03 Jan 2022 17:04

Repository Staff Only: item control page