CaltechAUTHORS
  A Caltech Library Service

Rayleigh EigenDirections (REDs): GAN latent space traversals for multidimensional features

Balakrishnan, Guha and Gadde, Raghudeep and Martinez, Aleix and Perona, Pietro (2022) Rayleigh EigenDirections (REDs): GAN latent space traversals for multidimensional features. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20220224-200918111

[img] PDF - Submitted Version
Creative Commons Attribution.

37MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20220224-200918111

Abstract

We present a method for finding paths in a deep generative model's latent space that can maximally vary one set of image features while holding others constant. Crucially, unlike past traversal approaches, ours can manipulate multidimensional features of an image such as facial identity and pixels within a specified region. Our method is principled and conceptually simple: optimal traversal directions are chosen by maximizing differential changes to one feature set such that changes to another set are negligible. We show that this problem is nearly equivalent to one of Rayleigh quotient maximization, and provide a closed-form solution to it based on solving a generalized eigenvalue equation. We use repeated computations of the corresponding optimal directions, which we call Rayleigh EigenDirections (REDs), to generate appropriately curved paths in latent space. We empirically evaluate our method using StyleGAN2 on two image domains: faces and living rooms. We show that our method is capable of controlling various multidimensional features out of the scope of previous latent space traversal methods: face identity, spatial frequency bands, pixels within a region, and the appearance and position of an object. Our work suggests that a wealth of opportunities lies in the local analysis of the geometry and semantics of latent spaces.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2201.10423arXivDiscussion Paper
ORCID:
AuthorORCID
Perona, Pietro0000-0002-7583-5809
Additional Information:Attribution 4.0 International (CC BY 4.0).
Record Number:CaltechAUTHORS:20220224-200918111
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20220224-200918111
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:113599
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:01 Mar 2022 19:14
Last Modified:01 Mar 2022 19:14

Repository Staff Only: item control page