A Caltech Library Service

Accelerating 3D Deep Learning with PyTorch3D

Ravi, Nikhila and Reizenstein, Jeremy and Novotny, David and Gordon, Taylor and Lo, Wan-Yen and Johnson, Justin and Gkioxari, Georgia (2020) Accelerating 3D Deep Learning with PyTorch3D. . (Unpublished)

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


Deep learning has significantly improved 2D image recognition. Extending into 3D may advance many new applications including autonomous vehicles, virtual and augmented reality, authoring 3D content, and even improving 2D recognition. However despite growing interest, 3D deep learning remains relatively underexplored. We believe that some of this disparity is due to the engineering challenges involved in 3D deep learning, such as efficiently processing heterogeneous data and reframing graphics operations to be differentiable. We address these challenges by introducing PyTorch3D, a library of modular, efficient, and differentiable operators for 3D deep learning. It includes a fast, modular differentiable renderer for meshes and point clouds, enabling analysis-by-synthesis approaches. Compared with other differentiable renderers, PyTorch3D is more modular and efficient, allowing users to more easily extend it while also gracefully scaling to large meshes and images. We compare the PyTorch3D operators and renderer with other implementations and demonstrate significant speed and memory improvements. We also use PyTorch3D to improve the state-of-the-art for unsupervised 3D mesh and point cloud prediction from 2D images on ShapeNet. PyTorch3D is open-source and we hope it will help accelerate research in 3D deep learning.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Ravi, Nikhila0000-0003-0097-5222
Johnson, Justin0000-0002-1251-088X
Record Number:CaltechAUTHORS:20221219-204802712
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:118418
Deposited By: George Porter
Deposited On:20 Dec 2022 04:09
Last Modified:02 Jun 2023 01:09

Repository Staff Only: item control page