Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published December 8, 2021 | Supplemental Material + Submitted + Published
Journal Article Open

Learning efficient navigation in vortical flow fields


Efficient point-to-point navigation in the presence of a background flow field is important for robotic applications such as ocean surveying. In such applications, robots may only have knowledge of their immediate surroundings or be faced with time-varying currents, which limits the use of optimal control techniques. Here, we apply a recently introduced Reinforcement Learning algorithm to discover time-efficient navigation policies to steer a fixed-speed swimmer through unsteady two-dimensional flow fields. The algorithm entails inputting environmental cues into a deep neural network that determines the swimmer's actions, and deploying Remember and Forget Experience Replay. We find that the resulting swimmers successfully exploit the background flow to reach the target, but that this success depends on the sensed environmental cue. Surprisingly, a velocity sensing approach significantly outperformed a bio-mimetic vorticity sensing approach, and achieved a near 100% success rate in reaching the target locations while approaching the time-efficiency of optimal navigation trajectories.

Additional Information

© The Author(s) 2021. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. Received 04 March 2021. Accepted 01 November 2021. Published 08 December 2021. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 1745301. P.G. was supported by this fellowship. Data availability. All data generated and discussed in this study are available within the article and its supplementary files, or are available from the authors upon request. Code availability. The Deep Reinforcement Learning algorithm V-RACER is available at github.com/cselab/smarties. Contributions. P.G., I.M., G.N., P.K., and J.O.D. designed research and were involved in discussions to interpret the results; P.G. performed research and analyzed results; G.N. and P.K. developed the V-RACER algorithm; G.N. wrote the software implementation of V-RACER; I.M. simulated the cylinder flow field; P.G. drafted the paper, and all authors helped edit and review. The authors declare no competing interests.

Attached Files

Published - s41467-021-27015-y.pdf

Submitted - 2102.10536.pdf

Supplemental Material - 41467_2021_27015_MOESM1_ESM.pdf

Supplemental Material - 41467_2021_27015_MOESM2_ESM.pdf

Supplemental Material - 41467_2021_27015_MOESM3_ESM.mp4

Supplemental Material - 41467_2021_27015_MOESM4_ESM.mp4


Files (63.1 MB)
Name Size Download all
23.2 MB Download
11.2 MB Preview Download
80.6 kB Preview Download
16.2 MB Download
1.8 MB Preview Download
10.7 MB Preview Download

Additional details

August 22, 2023
October 23, 2023