CaltechAUTHORS
  A Caltech Library Service

Automated Synthetic-to-Real Generalization

Chen, Wuyang and Yu, Zhiding and Wang, Zhangyang and Anandkumar, Anima (2020) Automated Synthetic-to-Real Generalization. Proceedings of Machine Learning Research, 119 . pp. 1746-1756. ISSN 2640-3498. https://resolver.caltech.edu/CaltechAUTHORS:20201106-120205331

[img] PDF - Published Version
See Usage Policy.

4MB
[img] PDF - Accepted Version
See Usage Policy.

4MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20201106-120205331

Abstract

Models trained on synthetic images often face degraded generalization to real data. As a convention, these models are often initialized with ImageNet pretrained representation. Yet the role of ImageNet knowledge is seldom discussed despite common practices that leverage this knowledge to maintain the generalization ability. An example is the careful hand-tuning of early stopping and layer-wise learning rates, which is shown to improve synthetic-to-real generalization but is also laborious and heuristic. In this work, we explicitly encourage the synthetically trained model to maintain similar representations with the ImageNet pretrained model, and propose a learning-to-optimize (L2O) strategy to automate the selection of layer-wise learning rates. We demonstrate that the proposed framework can significantly improve the synthetic-to-real generalization performance without seeing and training on real data, while also benefiting downstream tasks such as domain adaptation. Code is available at: https://github.com/NVlabs/ASG.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://proceedings.mlr.press/v119/chen20x.htmlPublisherArticle
https://arxiv.org/abs/2007.06965arXivDiscussion Paper
https://github.com/NVlabs/ASGRelated ItemCode
ORCID:
AuthorORCID
Wang, Zhangyang0000-0002-2050-5693
Additional Information:© 2020 by the author(s). Work done during internship at NVIDIA. We appreciate the computing power supported by NVIDIA GPU infrastructure. We also thank for the discussion and suggestions from four anonymous reviewers and the help from Yang Zou for the domain adaptation experiments. The research of Z. Wang was partially supported by NSF Award RI-1755701.
Funders:
Funding AgencyGrant Number
NSFIIS-1755701
Record Number:CaltechAUTHORS:20201106-120205331
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20201106-120205331
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:106487
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:06 Nov 2020 22:24
Last Modified:16 Sep 2021 17:07

Repository Staff Only: item control page