CaltechAUTHORS
  A Caltech Library Service

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning

Hong, Weizhe and Kennedy, Ann and Burgos-Artizzu, Xavier P. and Zelikowsky, Moriel and Navonne, Santiago G. and Perona, Pietro and Anderson, David J. (2015) Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning. Proceedings of the National Academy of Sciences, 112 (38). E5351-E5360. ISSN 0027-8424. PMCID PMC4603510. http://resolver.caltech.edu/CaltechAUTHORS:20150914-091025001

[img] PDF - Published Version
See Usage Policy.

2500Kb
[img] PDF - Supplemental Material
See Usage Policy.

1414Kb
[img] Video (MPEG) (Movie S1) - Supplemental Material
See Usage Policy.

3680Kb
[img] Video (MPEG) (Movie S2) - Supplemental Material
See Usage Policy.

5Mb
[img] Video (MPEG) (Movie S3) - Supplemental Material
See Usage Policy.

4013Kb
[img] Video (MPEG) (Movie S4) - Supplemental Material
See Usage Policy.

3802Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20150914-091025001

Abstract

A lack of automated, quantitative, and accurate assessment of social behaviors in mammalian animal models has limited progress toward understanding mechanisms underlying social interactions and their disorders such as autism. Here we present a new integrated hardware and software system that combines video tracking, depth sensing, and machine learning for automatic detection and quantification of social behaviors involving close and dynamic interactions between two mice of different coat colors in their home cage. We designed a hardware setup that integrates traditional video cameras with a depth camera, developed computer vision tools to extract the body “pose” of individual animals in a social context, and used a supervised learning algorithm to classify several well-described social behaviors. We validated the robustness of the automated classifiers in various experimental settings and used them to examine how genetic background, such as that of Black and Tan Brachyury (BTBR) mice (a previously reported autism model), influences social behavior. Our integrated approach allows for rapid, automated measurement of social behaviors across diverse experimental designs and also affords the ability to develop new, objective behavioral metrics.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1073/pnas.1515982112DOIArticle
http://www.pnas.org/content/112/38/E5351.abstractPublisherArticle
http://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1515982112/-/DCSupplementalPublisherSupporting Information
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4603510/PubMed CentralArticle
ORCID:
AuthorORCID
Perona, Pietro0000-0002-7583-5809
Additional Information:© 2015 National Academy of Sciences. Contributed by David J. Anderson, August 16, 2015 (sent for review May 20, 2015). Published online before print September 9, 2015. We thank Xiao Wang and Xiaolin Da for manual video annotation; Michele Damian, Louise Naud, and Robert Robertson for assistance with coding; Allan Wong for helpful suggestions; Prof. Sandeep R. Datta (Harvard University) for sharing unpublished data; Celine Chiu for laboratory management; and Gina Mancuso for administrative assistance. This work was supported by grants from the Moore and Simons Foundations and postdoctoral support from the Helen Hay Whitney Foundation (W.H.), the National Science Foundation (M.Z.) and Sloan-Swartz Foundation (A.K.). D.J.A. is an Investigator of the Howard Hughes Medical Institute. Author contributions: W.H., P.P., and D.J.A. designed research; W.H. performed research; W.H., X.P.B.-A., and S.G.N. contributed new reagents/analytic tools; W.H., A.K., M.Z., P.P., and D.J.A. analyzed data; and W.H., A.K., M.Z., P.P., and D.J.A. wrote the paper. The authors declare no conflict of interest. This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1515982112/-/DCSupplemental.
Funders:
Funding AgencyGrant Number
Gordon and Betty Moore FoundationUNSPECIFIED
Simons FoundationUNSPECIFIED
Helen Hay Whitney FoundationUNSPECIFIED
NSFUNSPECIFIED
Sloan-Swartz FoundationUNSPECIFIED
Howard Hughes Medical Institute (HHMI)UNSPECIFIED
Subject Keywords:social behavior; behavioral tracking; machine vision; depth sensing; supervised machine learning
PubMed Central ID:PMC4603510
Record Number:CaltechAUTHORS:20150914-091025001
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20150914-091025001
Official Citation:Weizhe Hong, Ann Kennedy, Xavier P. Burgos-Artizzu, Moriel Zelikowsky, Santiago G. Navonne, Pietro Perona, and David J. Anderson Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning PNAS 2015 112 (38) E5351-E5360; published ahead of print September 9, 2015, doi:10.1073/pnas.1515982112
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:60222
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:14 Sep 2015 17:13
Last Modified:11 Dec 2015 20:55

Repository Staff Only: item control page