A Caltech Library Service

Hierarchical Scene Annotation

Maire, Michael and Yu, Stella X. and Perona, Pietro (2013) Hierarchical Scene Annotation. In: Proceedings of the British Machine Vision Conference 2013. BMVA Press , Durham, UK, Art. No. 84. ISBN 1901725499.

[img] PDF - Published Version
See Usage Policy.


Use this Persistent URL to link to this item:


We present a computer-assisted annotation system, together with a labeled dataset and benchmark suite, for evaluating an algorithm’s ability to recover hierarchical scene structure. We evolve segmentation groundtruth from the two-dimensional image partition into a tree model that captures both occlusion and object-part relationships among possibly overlapping regions. Our tree model extends the segmentation problem to encompass object detection, object-part containment, and figure-ground ordering. We mitigate the cost of providing richer groundtruth labeling through a new web-based annotation tool with an intuitive graphical interface for rearranging the region hierarchy. Using precomputed superpixels, our tool also guides creation of user-specified regions with pixel-perfect boundaries. Widespread adoption of this human-machine combination should make the inaccuracies of bounding box labeling a relic of the past. Evaluating the state-of-the-art in fully automatic image segmentation reveals that it produces accurate two-dimension partitions, but does not respect groundtruth object-part structure. Our dataset and benchmark is the first to quantify these inadequacies. We illuminate recovery of rich scene structure as an important new goal for segmentation.

Item Type:Book Section
Related URLs:
URLURL TypeDescription
Perona, Pietro0000-0002-7583-5809
Additional Information:© 2013. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms. ONR MURI N00014-10-1-0933 and ARO/JPL-NASA Stennis NAS7.03001 supported this work. Part of Stella Yu’s work was supported by NSF CAREER IIS-1257700. Thanks to Alex Jose and Piotr Dollar for helpful discussion on user interfaces for segmentation
Funding AgencyGrant Number
Office of Naval Research (ONR)N00014-10-1-0933
Army Research Office (ARO)UNSPECIFIED
Record Number:CaltechAUTHORS:20190328-154052779
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:94260
Deposited By: George Porter
Deposited On:28 Mar 2019 22:59
Last Modified:16 Nov 2021 17:03

Repository Staff Only: item control page