A Caltech Library Service

The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting

Kay, Justin and Kulits, Peter and Stathatos, Suzanne and Deng, Siqi and Young, Erik and Beery, Sara and Van Horn, Grant and Perona, Pietro (2022) The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting. . (Unpublished)

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item:


We present the Caltech Fish Counting Dataset (CFC), a large-scale dataset for detecting, tracking, and counting fish in sonar videos. We identify sonar videos as a rich source of data for advancing low signal-to-noise computer vision applications and tackling domain generalization in multiple-object tracking (MOT) and counting. In comparison to existing MOT and counting datasets, which are largely restricted to videos of people and vehicles in cities, CFC is sourced from a natural-world domain where targets are not easily resolvable and appearance features cannot be easily leveraged for target re-identification. With over half a million annotations in over 1,500 videos sourced from seven different sonar cameras, CFC allows researchers to train MOT and counting algorithms and evaluate generalization performance at unseen test locations. We perform extensive baseline experiments and identify key challenges and opportunities for advancing the state of the art in generalization in MOT and counting.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Stathatos, Suzanne0000-0003-4351-4389
Young, Erik0000-0002-6783-2608
Beery, Sara0000-0002-2544-1844
Van Horn, Grant0000-0003-2953-9651
Perona, Pietro0000-0002-7583-5809
Additional Information:We are grateful to AWS for a gift to Trout Unlimited (TU) that supported data annotations, computational and storage costs, and to the Resnick Sustainability Institute at Caltech for funding to SB and PP. An NSF Fellowship supported SB. JK, SD, and EY volunteered their time. GVH was supported by the Macaulay Library at Cornell University. For collecting the dataset, and for feedback, encouragement, and moral support, we are grateful to: George Pess and Oleksandr Stefankiv (Northwest Fisheries Science Center); James Miller, Carl Pfisterer, Dawn Wilburn, Brandon Key, Suzanne Maxwell, Gregory Buck, April Faulkner, and Jordan Head (Alaska Department of Fish and Game); Dave Kajtaniak and Michael Sparkman (California Department of Fish and Wildlife); Dean Finnerty (TU’s Wild Steelhead Project); and Keith Denton, Mike McHenry, and the Lower Elwha Klallam Tribe.
Group:Resnick Sustainability Institute
Funding AgencyGrant Number
Amazon Web ServicesUNSPECIFIED
Resnick Sustainability InstituteUNSPECIFIED
NSF Graduate Research FellowshipUNSPECIFIED
Cornell UniversityUNSPECIFIED
Record Number:CaltechAUTHORS:20221219-234031928
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:118459
Deposited By: George Porter
Deposited On:21 Dec 2022 00:25
Last Modified:02 Jun 2023 01:28

Repository Staff Only: item control page