CaltechAUTHORS
  A Caltech Library Service

Boundary Detection Benchmarking: Beyond F-Measures

Hou, Xiaodi and Yuille, Alan and Koch, Christof (2013) Boundary Detection Benchmarking: Beyond F-Measures. In: 2013 Computer Vision and Pattern Recognition (CVPR). IEEE Conference on Computer Vision and Pattern Recognition. IEEE , New York, NY, pp. 2123-2130. ISBN 978-0-7695-4989-7. http://resolver.caltech.edu/CaltechAUTHORS:20140324-115812356

[img]
Preview
PDF - Submitted Version
See Usage Policy.

7Mb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20140324-115812356

Abstract

For an ill-posed problem like boundary detection, human labeled datasets play a critical role. Compared with the active research on finding a better boundary detector to refresh the performance record, there is surprisingly little discussion on the boundary detection benchmark itself. The goal of this paper is to identify the potential pitfalls of today's most popular boundary benchmark, BSDS 300. In the paper, we first introduce a psychophysical experiment to show that many of the "weak" boundary labels are unreliable and may contaminate the benchmark. Then we analyze the computation of f-measure and point out that the current benchmarking protocol encourages an algorithm to bias towards those problematic "weak" boundary labels. With this evidence, we focus on a new problem of detecting strong boundaries as one alternative. Finally, we assess the performances of 9 major algorithms on different ways of utilizing the dataset, suggesting new directions for improvements.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1109/CVPR.2013.276DOIArticle
http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6619120PublisherArticle
ORCID:
AuthorORCID
Koch, Christof0000-0001-6482-8067
Additional Information:© 2013 IEEE. Date of Conference: 23-28 June 2013. The first author would like to thank Zhuowen Tu, Yin Li and Liwei Wang for their thoughtful discussions. The research was supported by the ONR via an award made through Johns Hopkins University, by the G. Harold & Leila Y. Mathers Charitable Foundation, by Army Research Lab with 62250-CS and the Office of Naval Research N00014-12-10883. INSPEC Accession Number: 13824352.
Group:Koch Laboratory, KLAB
Funders:
Funding AgencyGrant Number
G. Harold and Leila Y. Mathers Charitable FoundationUNSPECIFIED
Army Research Laboratory62250-CS
Office of Naval Research (ONR)N000014-12-10883
Record Number:CaltechAUTHORS:20140324-115812356
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20140324-115812356
Official Citation:Xiaodi Hou; Yuille, A.; Koch, C., "Boundary Detection Benchmarking: Beyond F-Measures," Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on , vol., no., pp.2123,2130, 23-28 June 2013 doi: 10.1109/CVPR.2013.276
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:44468
Collection:CaltechAUTHORS
Deposited By: Ruth Sustaita
Deposited On:26 Mar 2014 21:15
Last Modified:20 Nov 2017 19:02

Repository Staff Only: item control page