A Caltech Library Service

Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild

Brazil, Garrick and Straub, Julian and Ravi, Nikhila and Johnson, Justin and Gkioxari, Georgia (2022) Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild. . (Unpublished)

[img] PDF - Submitted Version
Creative Commons Attribution Non-commercial Share Alike.


Use this Persistent URL to link to this item:


Recognizing scenes and objects in 3D from a single image is a longstanding goal of computer vision with applications in robotics and AR/VR. For 2D recognition, large datasets and scalable solutions have led to unprecedented advances. In 3D, existing benchmarks are small in size and approaches specialize in few object categories and specific domains, e.g. urban driving scenes. Motivated by the success of 2D recognition, we revisit the task of 3D object detection by introducing a large benchmark, called Omni3D. Omni3D re-purposes and combines existing datasets resulting in 234k images annotated with more than 3 million instances and 97 categories.3D detection at such scale is challenging due to variations in camera intrinsics and the rich diversity of scene and object types. We propose a model, called Cube R-CNN, designed to generalize across camera and scene types with a unified approach. We show that Cube R-CNN outperforms prior works on the larger Omni3D and existing benchmarks. Finally, we prove that Omni3D is a powerful dataset for 3D object recognition, show that it improves single-dataset performance and can accelerate learning on new smaller datasets via pre-training.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Ravi, Nikhila0000-0003-0097-5222
Johnson, Justin0000-0002-1251-088X
Additional Information:Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).
Record Number:CaltechAUTHORS:20221219-204749212
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:118410
Deposited By: George Porter
Deposited On:20 Dec 2022 03:50
Last Modified:20 Dec 2022 03:50

Repository Staff Only: item control page