Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published September 22, 2015 | Published + Supplemental Material
Journal Article Open

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning


A lack of automated, quantitative, and accurate assessment of social behaviors in mammalian animal models has limited progress toward understanding mechanisms underlying social interactions and their disorders such as autism. Here we present a new integrated hardware and software system that combines video tracking, depth sensing, and machine learning for automatic detection and quantification of social behaviors involving close and dynamic interactions between two mice of different coat colors in their home cage. We designed a hardware setup that integrates traditional video cameras with a depth camera, developed computer vision tools to extract the body "pose" of individual animals in a social context, and used a supervised learning algorithm to classify several well-described social behaviors. We validated the robustness of the automated classifiers in various experimental settings and used them to examine how genetic background, such as that of Black and Tan Brachyury (BTBR) mice (a previously reported autism model), influences social behavior. Our integrated approach allows for rapid, automated measurement of social behaviors across diverse experimental designs and also affords the ability to develop new, objective behavioral metrics.

Additional Information

© 2015 National Academy of Sciences. Contributed by David J. Anderson, August 16, 2015 (sent for review May 20, 2015). Published online before print September 9, 2015. We thank Xiao Wang and Xiaolin Da for manual video annotation; Michele Damian, Louise Naud, and Robert Robertson for assistance with coding; Allan Wong for helpful suggestions; Prof. Sandeep R. Datta (Harvard University) for sharing unpublished data; Celine Chiu for laboratory management; and Gina Mancuso for administrative assistance. This work was supported by grants from the Moore and Simons Foundations and postdoctoral support from the Helen Hay Whitney Foundation (W.H.), the National Science Foundation (M.Z.) and Sloan-Swartz Foundation (A.K.). D.J.A. is an Investigator of the Howard Hughes Medical Institute. Author contributions: W.H., P.P., and D.J.A. designed research; W.H. performed research; W.H., X.P.B.-A., and S.G.N. contributed new reagents/analytic tools; W.H., A.K., M.Z., P.P., and D.J.A. analyzed data; and W.H., A.K., M.Z., P.P., and D.J.A. wrote the paper. The authors declare no conflict of interest. This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1515982112/-/DCSupplemental.

Attached Files

Published - PNAS-2015-Hong-E5351-60.pdf

Supplemental Material - pnas.1515982112.sm01.mp4

Supplemental Material - pnas.1515982112.sm02.mp4

Supplemental Material - pnas.1515982112.sm03.mp4

Supplemental Material - pnas.1515982112.sm04.mp4

Supplemental Material - pnas.201515982SI.pdf


Files (21.4 MB)
Name Size Download all
3.8 MB Download
1.4 MB Preview Download
4.1 MB Download
5.7 MB Download
2.6 MB Preview Download
3.9 MB Download

Additional details

August 22, 2023
October 24, 2023