Hou, Xiaodi and Yuille, Alan and Koch, Christof (2013) Boundary Detection Benchmarking: Beyond F-Measures. In: 2013 Computer Vision and Pattern Recognition (CVPR). IEEE Conference on Computer Vision and Pattern Recognition. IEEE , New York, NY, pp. 2123-2130. ISBN 978-0-7695-4989-7. http://resolver.caltech.edu/CaltechAUTHORS:20140324-115812356
|
PDF
- Submitted Version
See Usage Policy. 7Mb |
Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20140324-115812356
Abstract
For an ill-posed problem like boundary detection, human labeled datasets play a critical role. Compared with the active research on finding a better boundary detector to refresh the performance record, there is surprisingly little discussion on the boundary detection benchmark itself. The goal of this paper is to identify the potential pitfalls of today's most popular boundary benchmark, BSDS 300. In the paper, we first introduce a psychophysical experiment to show that many of the "weak" boundary labels are unreliable and may contaminate the benchmark. Then we analyze the computation of f-measure and point out that the current benchmarking protocol encourages an algorithm to bias towards those problematic "weak" boundary labels. With this evidence, we focus on a new problem of detecting strong boundaries as one alternative. Finally, we assess the performances of 9 major algorithms on different ways of utilizing the dataset, suggesting new directions for improvements.
| Item Type: | Book Section | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Related URLs: |
| |||||||||
| ORCID: |
| |||||||||
| Additional Information: | © 2013 IEEE. Date of Conference: 23-28 June 2013. The first author would like to thank Zhuowen Tu, Yin Li and Liwei Wang for their thoughtful discussions. The research was supported by the ONR via an award made through Johns Hopkins University, by the G. Harold & Leila Y. Mathers Charitable Foundation, by Army Research Lab with 62250-CS and the Office of Naval Research N00014-12-10883. INSPEC Accession Number: 13824352. | |||||||||
| Group: | Koch Laboratory (KLAB) | |||||||||
| Funders: |
| |||||||||
| Record Number: | CaltechAUTHORS:20140324-115812356 | |||||||||
| Persistent URL: | http://resolver.caltech.edu/CaltechAUTHORS:20140324-115812356 | |||||||||
| Official Citation: | Xiaodi Hou; Yuille, A.; Koch, C., "Boundary Detection Benchmarking: Beyond F-Measures," Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on , vol., no., pp.2123,2130, 23-28 June 2013 doi: 10.1109/CVPR.2013.276 | |||||||||
| Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | |||||||||
| ID Code: | 44468 | |||||||||
| Collection: | CaltechAUTHORS | |||||||||
| Deposited By: | Ruth Sustaita | |||||||||
| Deposited On: | 26 Mar 2014 21:15 | |||||||||
| Last Modified: | 25 Oct 2018 21:04 |
Repository Staff Only: item control page
