CaltechAUTHORS
  A Caltech Library Service

SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies

Fan, Linxi and Wang, Guanzhi and Huang, De-An and Yu, Zhiding and Fei-Fei, Li and Zhu, Yuke and Anandkumar, Anima (2021) SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies. Proceedings of Machine Learning Research, 139 . pp. 3088-3099. ISSN 2640-3498. https://resolver.caltech.edu/CaltechAUTHORS:20210831-203918113

[img] PDF - Published Version
See Usage Policy.

9MB
[img] PDF - Accepted Version
See Usage Policy.

9MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20210831-203918113

Abstract

Generalization has been a long-standing challenge for reinforcement learning (RL). Visual RL, in particular, can be easily distracted by irrelevant factors in high-dimensional observation space. In this work, we consider robust policy learning which targets zero-shot generalization to unseen visual environments with large distributional shift. We propose SECANT, a novel self-expert cloning technique that leverages image augmentation in two stages to *decouple* robust representation learning from policy optimization. Specifically, an expert policy is first trained by RL from scratch with weak augmentations. A student network then learns to mimic the expert policy by supervised learning with strong augmentations, making its representation more robust against visual variations compared to the expert. Extensive experiments demonstrate that SECANT significantly advances the state of the art in zero-shot generalization across 4 challenging domains. Our average reward improvements over prior SOTAs are: DeepMind Control (+26.5%), robotic manipulation (+337.8%), vision-based autonomous driving (+47.7%), and indoor object navigation (+15.8%). Code release and video are available at https://linxifan.github.io/secant-site/.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://proceedings.mlr.press/v139/fan21c.htmlPublisherArticle
http://arxiv.org/abs/2106.09678arXivDiscussion Paper
https://linxifan.github.io/secant-site/Related ItemCode release and video
ORCID:
AuthorORCID
Zhu, Yuke0000-0002-9198-2227
Additional Information:© 2021 by the author(s). We are extremely grateful to Chris Choy, Jean Kossaifi, Shikun Liu, Zhiyuan “Jerry” Lin, Josiah Wong, Huaizu Jiang, Guanya Shi, Jacob Austin, Ismail Elezi, Ajay Mandlekar, Fei Xia, Agrim Gupta, Shyamal Buch, and many other colleagues for their helpful feedback and insightful discussions.
Record Number:CaltechAUTHORS:20210831-203918113
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20210831-203918113
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:110651
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:01 Sep 2021 14:42
Last Modified:16 Sep 2021 17:06

Repository Staff Only: item control page