CaltechAUTHORS
  A Caltech Library Service

Distributionally Robust Learning for Unsupervised Domain Adaptation

Wang, Haoxuan and Liu, Anqi and Yu, Zhiding and Yue, Yisong and Anandkumar, Anima (2020) Distributionally Robust Learning for Unsupervised Domain Adaptation. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20201106-120148344

[img] PDF - Submitted Version
See Usage Policy.

6MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20201106-120148344

Abstract

We propose a distributionally robust learning (DRL) method for unsupervised domain adaptation (UDA) that scales to modern computer vision benchmarks. DRL can be naturally formulated as a competitive two-player game between a predictor and an adversary that is allowed to corrupt the labels, subject to certain constraints, and reduces to incorporating a density ratio between the source and target domains (under the standard log loss). This formulation motivates the use of two neural networks that are jointly trained - a discriminative network between the source and target domains for density-ratio estimation, in addition to the standard classification network. The use of a density ratio in DRL prevents the model from being overconfident on target inputs far away from the source domain. Thus, DRL provides conservative confidence estimation in the target domain, even when the target labels are not available. This conservatism motivates the use of DRL in self-training for sample selection, and we term the approach distributionally robust self-training (DRST). In our experiments, DRST generates more calibrated probabilities and achieves state-of-the-art self-training accuracy on benchmark datasets. We demonstrate that DRST captures shape features more effectively, and reduces the extent of distributional shift during self-training.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2010.05784arXivDiscussion Paper
ORCID:
AuthorORCID
Yue, Yisong0000-0001-9127-1989
Record Number:CaltechAUTHORS:20201106-120148344
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20201106-120148344
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:106482
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:06 Nov 2020 20:45
Last Modified:06 Nov 2020 20:45

Repository Staff Only: item control page