CaltechAUTHORS
  A Caltech Library Service

Multiview Compressive Coding for 3D Reconstruction

Wu, Chao-Yuan and Johnson, Justin and Malik, Jitendra and Feichtenhofer, Christoph and Gkioxari, Georgia (2023) Multiview Compressive Coding for 3D Reconstruction. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20230316-204045919

[img] PDF - Submitted Version
See Usage Policy.

13MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20230316-204045919

Abstract

A central goal of visual recognition is to understand objects and scenes from a single image. 2D recognition has witnessed tremendous progress thanks to large-scale learning and general-purpose representations. Comparatively, 3D poses new challenges stemming from occlusions not depicted in the image. Prior works try to overcome these by inferring from multiple views or rely on scarce CAD models and category-specific priors which hinder scaling to novel settings. In this work, we explore single-view 3D reconstruction by learning generalizable representations inspired by advances in self-supervised learning. We introduce a simple framework that operates on 3D points of single objects or whole scenes coupled with category-agnostic large-scale training from diverse RGB-D videos. Our model, Multiview Compressive Coding (MCC), learns to compress the input appearance and geometry to predict the 3D structure by querying a 3D-aware decoder. MCC's generality and efficiency allow it to learn from large-scale and diverse data sources with strong generalization to novel objects imagined by DALL⋅E 2 or captured in-the-wild with an iPhone.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2301.08247arXivDiscussion Paper
https://mcc3d.github.io/Related ItemProject website
ORCID:
AuthorORCID
Wu, Chao-Yuan0000-0002-5690-8865
Johnson, Justin0000-0002-1251-088X
Malik, Jitendra0000-0003-3695-1580
Feichtenhofer, Christoph0000-0001-9756-7238
Record Number:CaltechAUTHORS:20230316-204045919
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20230316-204045919
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:120106
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:16 Mar 2023 23:21
Last Modified:16 Mar 2023 23:21

Repository Staff Only: item control page