A Caltech Library Service

Multi-camera real-time three-dimensional tracking of multiple flying animals

Straw, Andrew D. and Branson, Kristin and Neumann, Titus R. and Dickinson, Michael H. (2011) Multi-camera real-time three-dimensional tracking of multiple flying animals. Journal of the Royal Society Interface, 8 (56). pp. 395-409. ISSN 1742-5689. PMCID PMC3030815.

PDF - Published Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Automated tracking of animal movement allows analyses that would not otherwise be possible by providing great quantities of data. The additional capability of tracking in real time—with minimal latency—opens up the experimental possibility of manipulating sensory feedback, thus allowing detailed explorations of the neural basis for control of behaviour. Here, we describe a system capable of tracking the three-dimensional position and body orientation of animals such as flies and birds. The system operates with less than 40 ms latency and can track multiple animals simultaneously. To achieve these results, a multi-target tracking algorithm was developed based on the extended Kalman filter and the nearest neighbour standard filter data association algorithm. In one implementation, an 11-camera system is capable of tracking three flies simultaneously at 60 frames per second using a gigabit network of nine standard Intel Pentium 4 and Core 2 Duo computers. This manuscript presents the rationale and details of the algorithms employed and shows three implementations of the system. An experiment was performed using the tracking system to measure the effect of visual contrast on the flight speed of Drosophila melanogaster. At low contrasts, speed is more variable and faster on average than at high contrasts. Thus, the system is already a useful tool to study the neurobiology and behaviour of freely flying animals. If combined with other techniques, such as ‘virtual reality’-type computer graphics or genetic manipulation, the tracking system would offer a powerful new way to investigate the biology of flying animals.

Item Type:Article
Related URLs:
URLURL TypeDescription DOIArticle CentralArticle
Straw, Andrew D.0000-0001-8381-0858
Branson, Kristin0000-0002-5567-2512
Dickinson, Michael H.0000-0002-8587-9936
Additional Information:© 2010 The Royal Society. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Received 19 April 2010. Accepted 21 June 2010. Published online 14 July 2010. The data for figure 4 were gathered in collaboration with Douglas Altshuler. Sawyer Fuller helped with the EKF formulation, provided helpful feedback on the manuscript and, together with Gaby Maimon, Rosalyn Sayaman, Martin Peek and Aza Raskin, helped with physical construction of arenas and bug reports on the software. Pete Trautmann provided insight on data association, and Pietro Perona provided helpful suggestions on the manuscript. This work was supported by grants from the Packard Foundation, AFOSR (FA9550-06-1-0079), ARO (DAAD 19-03-D-0004), NIH (R01 DA022777) and NSF (0923802) to M.H.D. and AFOSR (FA9550-10-1-0086) to A.D.S.
Funding AgencyGrant Number
David and Lucile Packard FoundationUNSPECIFIED
Air Force Office of Scientific Research (AFOSR)FA9550-06-1-0079
Army Research Office (ARO)DAAD 19-03-D-0004
Air Force Office of Scientific Research (AFOSR)FA9550-10-1-0086
Subject Keywords:computer vision; animal behaviour; flight; manoeuvring; insects; birds
Issue or Number:56
PubMed Central ID:PMC3030815
Record Number:CaltechAUTHORS:20110304-085606048
Persistent URL:
Official Citation:Andrew D. Straw, Kristin Branson, Titus R. Neumann, and Michael H. Dickinson Multi-camera real-time three-dimensional tracking of multiple flying animals J. R. Soc. Interface published online before print July 14, 2010, doi:10.1098/rsif.2010.0230
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:22657
Deposited By: Ruth Sustaita
Deposited On:04 Mar 2011 18:57
Last Modified:09 Mar 2020 13:18

Repository Staff Only: item control page