A Caltech Library Service

Boundary Detection Benchmarking: Beyond F-Measures

Hou, Xiaodi and Yuille, Alan and Koch, Christof (2013) Boundary Detection Benchmarking: Beyond F-Measures. In: 2013 Computer Vision and Pattern Recognition (CVPR). IEEE Conference on Computer Vision and Pattern Recognition. IEEE , New York, NY, pp. 2123-2130. ISBN 978-0-7695-4989-7.

PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


For an ill-posed problem like boundary detection, human labeled datasets play a critical role. Compared with the active research on finding a better boundary detector to refresh the performance record, there is surprisingly little discussion on the boundary detection benchmark itself. The goal of this paper is to identify the potential pitfalls of today's most popular boundary benchmark, BSDS 300. In the paper, we first introduce a psychophysical experiment to show that many of the "weak" boundary labels are unreliable and may contaminate the benchmark. Then we analyze the computation of f-measure and point out that the current benchmarking protocol encourages an algorithm to bias towards those problematic "weak" boundary labels. With this evidence, we focus on a new problem of detecting strong boundaries as one alternative. Finally, we assess the performances of 9 major algorithms on different ways of utilizing the dataset, suggesting new directions for improvements.

Item Type:Book Section
Related URLs:
URLURL TypeDescription
Koch, Christof0000-0001-6482-8067
Additional Information:© 2013 IEEE. Date of Conference: 23-28 June 2013. The first author would like to thank Zhuowen Tu, Yin Li and Liwei Wang for their thoughtful discussions. The research was supported by the ONR via an award made through Johns Hopkins University, by the G. Harold & Leila Y. Mathers Charitable Foundation, by Army Research Lab with 62250-CS and the Office of Naval Research N00014-12-10883. INSPEC Accession Number: 13824352.
Group:Koch Laboratory (KLAB)
Funding AgencyGrant Number
G. Harold and Leila Y. Mathers Charitable FoundationUNSPECIFIED
Army Research Laboratory62250-CS
Office of Naval Research (ONR)N000014-12-10883
Series Name:IEEE Conference on Computer Vision and Pattern Recognition
Record Number:CaltechAUTHORS:20140324-115812356
Persistent URL:
Official Citation:Xiaodi Hou; Yuille, A.; Koch, C., "Boundary Detection Benchmarking: Beyond F-Measures," Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on , vol., no., pp.2123,2130, 23-28 June 2013 doi: 10.1109/CVPR.2013.276
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:44468
Deposited By: Ruth Sustaita
Deposited On:26 Mar 2014 21:15
Last Modified:03 Oct 2019 06:18

Repository Staff Only: item control page