A Caltech Library Service

Implicit competitive regularization in GANs

Schäfer, Florian and Zheng, Hongkai and Anandkumar, Anima (2019) Implicit competitive regularization in GANs. . (Unpublished)

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


Generative adversarial networks (GANs) are capable of producing high quality samples, but they suffer from numerous issues such as instability and mode collapse during training. To combat this, we propose to model the generator and discriminator as agents acting under local information, uncertainty, and awareness of their opponent. By doing so we achieve stable convergence, even when the underlying game has no Nash equilibria. We call this mechanism implicit competitive regularization (ICR) and show that it is present in the recently proposed competitive gradient descent (CGD). When comparing CGD to Adam using a variety of loss functions and regularizers on CIFAR10, CGD shows a much more consistent performance, which we attribute to ICR. In our experiments, we achieve the highest inception score when using the WGAN loss (without gradient penalty or weight clipping) together with CGD. This can be interpreted as minimizing a form of integral probability metric based on ICR.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Additional Information:A. Anandkumar is supported in part by Bren endowed chair, DARPA PAIHR00111890035, Raytheon, and Microsoft, Google and Adobe faculty fellowships. F. Schäfer gratefully acknowledges support by the Air Force Office of Scientific Research under award number FA9550-18-1-0271 (Games for Computation and Learning). H. Zheng is supported by Zhiyuan College, Shanghai Jiao Tong University.
Funding AgencyGrant Number
Bren Professor of Computing and Mathematical SciencesUNSPECIFIED
Defense Advanced Research Projects Agency (DARPA)HR0011-18-9-0035
Air Force Office of Scientific Research (AFOSR)FA9550-18-1-0271
Shanghai Jiao Tong UniversityUNSPECIFIED
Record Number:CaltechAUTHORS:20200109-090758476
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:100580
Deposited By: Tony Diaz
Deposited On:09 Jan 2020 18:18
Last Modified:09 Jan 2020 18:18

Repository Staff Only: item control page