Microsoft Word - Table S1_NEW.docx

Supplementary

Material

Automated i

mag

based tracking and its application in ecology

Anthony I. Dell*, John A. Bender, Kristin Branson, Iain D. Couzin, Gonzalo G. de Polavieja, Lucas P.J.J. Noldus, Alfonso Pére

Escudero,

Pietro Perona, Andrew D. Straw, Martin Wikelski, and Ulrich Brose

*Corresponding author:

adell@gwdg.de

Table S

. Some

key automated image

based tracking systems (in alphabetic order) that are readily available to research ecologists,

with focus on those suited for tracking behavior during species interactions. See

Table S2

for detailed descriptions of each category.

3DTra

cker

[

]

(

Movie!S

)

Availability)and)usability:

open

source,!simple!to!use,!specific

Raw)output:

3D!trajectories

,!3D!pose

Requirements

Win,!Matlab,!

multiple!

D!cameras

Maximum)individuals:

Shape)

requirements:

rodent

Identity)method:

crosses!solved

Analysis)tools:

basic!kinematics,!social!behaviour

Taxa)studied:

rats

)

Required)resolution)per)individual:

300!points

Notes:)

Executables!and!sources!available!online!at!

http://matsumotoj.github.io/

[

]

Requires!

uncluttered

background.

Requires!multiple!3

D!cameras.!

Estimates!3D!trajectories!and!detailed!3D!pose!of!four!body!parts!(head,!

neck,!trunk,!and!hip)

Currently!uses!4!x!Microsoft!Kinect!cameras.!Matlab!

required!for!behavioral!analysis.!

Developed!for!rats,!but!could!be!applied!to!mice!with!higher!resolution!cameras.

3D!video!enables!more!stable!

tracking!during!close!contact.!On!average,!automation!makes!errors!in!20%!of!1!min!videos

Includes

tool

facilitate

manual

correction

Includes!Matlab!scripts!extracting!basic!movement!parameters!(such!as!velocity!or!

angle)!and!social!behaviors!(e.g.,!approaching,!mounting,!head

hip!contact)!based!on!3D!trajectories!of!body!parts

Social!interactions!in!rat

[

]

3D!points!covering!the!surface!of!each!individual

CADABRA

[

]

(

Movie!S

)

Availability)and)usability:

open

source,!simple!to!use,!flexible

Raw)output:

2D!trajectories,!detailed!pose

Requirements

Win/Linux/Mac,!Matlab

Maximum)individuals:

Shape)

requirements:

fly

shaped

Identity)method:

maintained!(size/markings)

Analysis)tools:

extensive

Taxa)studied:

flies

)

Required)resolution)per)individual:

20!pixels

Notes:)

Executables!available!online!

[

]

.!Source!code!available!upon!request.!Flexible!to!different!setups.!Can!use!conventional!cameras.!Can!track!several!arenas!si

multaneously.!Requires!constant!and!uncluttered!

background.!

Estimates!25!traits!of!the!pose!and!position!of!each!individual!(orien

tation,!velocity,!size,!wing!pose).!

Matlab!Compiler!Runtime!library!(free).

Developed!for!

Drosophila

Uses!size!difference!between!pairs!(e.g.,!

male!or!female).!Similar!sized!individuals!must!be!marked!(otherwise!system!makes!highest!probability!matches

Uses!estimates!of!25!traits!of!both!flies!(position,!orientation,!velocity,!size,!wing!pose)!to!automatically!

quantify!8!social!behavior!(e.g.,!lunging,!wing!extension,!copulation).

Genetic!and!environmental!influences!on!social!behavior!in!

Drosophila

[

]

Ctrax

[

]

(

Movie!S

)

Availability)and)usability:

open

source,!simple!to!use,!flexible

Raw)output:

2D!trajectories,!orientations

Requirements:

Win/Linux/Mac,!Matlab

Maximum)individuals:

Shape)

requirements:

elliptical

Identity)method:

crosses!solved

Analysis)tools:

extensive,!see!JAABA

Taxa)studied:

flies,!cockroaches,!fish,!spiders

)

Required)resolution)per)individual:

15!pixels!

Notes

)

Ctrax!online!at!

http://ctrax.sourceforge.net

.!Flexible!to!different!setups.!Can!use!conventional!cameras.!Requires!constant!and!uncluttered!background.!Ongoing!maintenan

ce,!enhancements,!and!support!provided.

Matlab!required!for

capabilities!beyond!tracking,!which!are!important!(i.e.,!error!fixing,!analysis).!

Developed!for!

Drosophila

,!works!well!with!any!species!of!elliptical!shape.!Individuals!must!have!similar!size.

For!

Drosophila

with!full!automation!system!makes!an!identit

y!error!on!average!once!every!5!fly

hours!with!density!of!10,!once!every!1.5!fly

hours!with!dentisy!of!20,!and!once!every!40!

minutes!with!density!of!50.!Error!rate!is!zero!with!

minimal!user!supervision.!Includes!application!to!facilitate!manual!correct

ion.

JAABA!

[

]

supersedes!Ctrax’s!behavioral!analysis!package!(free!online!at!

http://jaaba.sourceforge.net

).!JAABA

is!a!machine

learning!system!to!

create!automatic!behavior!classifiers,!which!allows!exploration!of!differences!in!thousands!of!behavior!statistics!between!la

rge!numbers!of!individuals.!JABBA!requires!tracking!data!from!other!tracking!systems!(e.g.,!Ctrax,

CADABRA,!Multi

Worm!Tracker,!Motr).

Genetic!and!environmental!influences!on!

Drosophila

social!behavior!

[

]

cockroach!locomotion!

[

]

,!collective!behavior!in!fish!

[

]

,!spider!foraging!behavior!(Dell!et!al.!unpublished).

EthoVision

)

(

Movie!S

)

Availability)and)usability

commercial,!simple!to!use,!flexible

)

Raw)output

2D!trajectories,!orientations

)

Requirements

Win!

)

Maximum)individuals

(note!3)

Shape)requirements

flexible

Identity)method

maintained!(size/markings)

)

Analysis)tools

extensive

)

Taxa)studied

rodents,!fish,!insects,!pigs,!spiders,!more

)

Required)resolution)per)individual

15!pixels

)

Notes

Developed!by!Noldus

Information!Technology!BV.!Available!online!at!

http://www.noldus.com/ethovision

Free!trial!version!available.!Software!only!or!with!integrated!hardware.!Valid!for!conventional!cameras!and!

laboratory!setups

,!including!multiple!arenas.!A!constant!and!uncluttered!background!not!required.!Shadows,!bedding!material!and!objects!can!be

present!in!the!arena.!Dedicated!systems!available!for!fish!(DanioVision)!and!

rodents!(PhenoTyper).!

Track3D!(

http://www.noldus.com/innovationworks/products/track3d

extension!allows!3D!tracking!of!single!individuals

using!2!cameras

.!Standard!software!tracks!center!of!body!mass,!specialized!

modules!recognize!body!s

hape!and!multiple!body!points!(e.g.!nose!point,!tail!base)!of!rodents.!Simple!and!easy

use!interface.!

Can!track!100!arenas!(16!individuals!in!each!arena)!with!a!single!camera.)!

Can!track!many!

shapes,!from!small!insects!to!large!mammals.!

Similar

sized!individuals!require!color!marking!(when!maintenance!of!identity!required).!With!

small!numbers!of!individuals!the!system!can!often!maintain!identities!of!unmarked!

animals.

Includes!visualization,!data!editing!and!animation!options.!Module!for!automa

ted!behavior!recognition!(rodents:!rearing,!grooming,!sniffing),!trial!and!hardware!control,!behavior

physiology!integration.

See!all!

publications!at!

http://www.noldus.com/ethovisio

xt/selected

publications

Requires!3!pixels!per!individual!for!position!only.

Flydra

)

[

]

(

Movie!S

)

Availability)and)usability

open

source,!difficult!to!use,!specific

)

Raw)output

3D!trajectories!(real

time)

)

Requirements

Linux,!multiple!cameras

)

Maximum)individuals

(note!2)

)

Shape)

requirements

flexible

Identity)method

not!maintained

)

Analysis)tools

none

)

Taxa)studied

flies,!birds

)

Required)resolution)per)individual

1!pixel

)

Notes:)

Core!3D!algorithms!available!

under!open

source!license!

at!

https://github.com/strawlab/pymvg

(image!acquisition!and!synchronization!code,!extended!kalman!filter!for!tracking,!and!analysis!tools!to!verify!tracking!is!

working!are!not!currently!open

source).

Requires!sy

nchronization!of!multiple!computers!and!high

speed!cameras.!Requires!constant!and!uncluttered!background!(some!degree!of!habitat!complexity!possible).!Advanced!Linux!

skills!needed.!

Limited!by!computer!speed.!

Individuals!should!be!roughly!the!same!size,!a

nd!have!independent!behavior!(i.e.!not!valid!for!social!species,!aggression,!courtship,!predation,!etc.).

Flight!control!in!

Drosophila

and!

hummingbirds!

[

]

GroupHousedScan

(

Movie!S

)

Availability)and)usability

commercial,!simple!to!use,!flexible

)

Raw)output

3D!trajectories,!detailed!pose!(real

time)

)

Requirements

Win,!two!cameras

)

Maximum)individuals

(note!4)

)

Shape)requirements

rodent

Identity)method

maintained!(size/markings)

)

Analysis)tools

extensive

)

Taxa)studied

rats,!mice,!primates

)

Required)resolution)per)individual

200!pixels

)

Notes

Developed!by!

CleverSys!Inc.!Available!online!at!

http://cleversysinc.com/?csi_products=grouphousedscan

.!Software!only!or!with!integrated!hardware.

Valid!for!conventional!cameras!and!laboratory!setups.!Requires!

contrasting!background.!Abi

lity!to!easily!characterize!spatial!components!of!the!physical!environment!for!integration!with!tracking!data.!Bedding!materi

al,!food!and!water!containers,!and!other!objects!can!be!present!in!the!

arena.!Useful!for!long

term!tracking!over!longer!durations,!

such!as!multiple!days,!as!is!capable!of!adjusting!between!day!and!night!conditions!automatically.!Simple!and!easy

use!interface.!

Dedicated!2D!systems!can!

monitor!the!movement!and!detailed!pose!of!single!animals!either!from!the!side!(HomeCageScan!

http://cleversysinc.com/?csi_products=homecagescan

and!PrimateScan!

http://cleversysinc.com/?csi_products=primatescan

)

or!top!

(TopScan!

http://cleversysinc.com/?csi_products=topscan

suite

).!TopScan!can!be!extended!to!4!individuals!with!SocialScan!add

on.!Pose!data!include!8!different!body!points!on!a!rodent:!hea

d!or!nose,!ears,!forelimbs,!

hindlimbs,!upper!back,!lower!back,!abdomen,!tail.!HomeCageScan!identifies!same!8!points.!PrimateScan!identies!same!8!points!o

n!a!primate.!TopScan!and!SocialScan!identify!4!points!on!a!rodent:!nose,!forelimb,!center!of!

mass,!tail

base.!

Single

camera!options!available!(see!note!2).!

see!note!2.!

Developed!for!mice!and!rats,!has!dedicated!system!for!a!single!primate!(see!note!2).!

Kinematic!measurements!such!as!speed,!velocity,!orientation,!

shape,!etc.!Automatically!outputs!individu

al!(

eat,!drink,!sleep,!walk,!jump,!rear!up,!hang,!groom,!sniff,!twitch,!stretch,!etc.)!

and!social!interaction!(contact,!sniff,!follow,!leave,!approach,!etc.)!behaviors!and!events

Genetic!and!

environmental!disorders!in!rats!and!mice!(unpublished).!

Can!be

set!as!a!threshold.

GroupScan

(

Movie!S

)

Availability)and)usability

commercial,!simple!to!use,!flexible

)

Raw)output

2D!trajectories

)

Requirements

Win

)

Maximum)individuals

100

(note!2)

)

Shape)requirements

flexible

Identity)

method

not!maintained

)

Analysis)tools

basic!kinematics

)

Taxa)studied

Drosophila

,!fish

)

Required)resolution)per)individual

10!pixels

)

Notes

Developed!by!

CleverSys!Inc.!Available!online!at!

http://cleversysinc.com/?csi_products=groupscan

.!Software!only!or!with!integrated!hardware.

Valid!for!conventional!cameras!and!laboratory!setups.!Requires!

contrasting!background.!Ability!to!easi

ly!characterize!spatial!components!of!the!physical!environment!for!integration!with!tracking!data.!Simple!and!easy

use!interface.!

Preset!at!100,!but!can!be!varied.!

Develope

d!for!fruit!

flies!and!fish,!but!extendible!to!any!species.!

Basic!population

vel!statistics,!including!count,!and!average!

distance!travelled,!velocity,!inter

frame!body!pixel!change,!etc.!

Genetic!and!environmental!disorders!in!

Drosophila

and!fish!(unpublished).!

Can!be!set!as!a!threshold.

)

idTracker

)

[

]

(

Movie!S

)

Availability)and)usability

open

source,!simple!to!use,!flexible

)

Raw)output

2D!trajectories,!orientations

)

Requirements

Win/Linux/Mac,!matlab

)

Maximum)individuals

(note!3)

)

Shape)requirements

flexible

)

Identity)method

maintained!(fingerprinting)

)

Analysis)tools

extensive,!see!idSocial

Taxa)studied

mice,!fish,!flies,!ants

Required)resolution)per)individual

150!pixels

)

Notes

Available!at!

www.idtracker.es

Simple!to!use.!Adaptable!to!different!laboratory!setups.!Requires!image!with!good!contrast!between!animals!and!background.!Va

lid!for!conventional!cameras.!Can!track!compressed!

videos.

Development!ver

sion!requires!Matlab!(free!compiled!binaries!do!not).!Fingerprinting!computationally!heavy,!runs!on!conventional!computer!wit

h!at!least!4GB!RAM!.

Depends!on!species!and!conditions.!

No!inherent!

contour.!Can!track!wide!range!of!size!and!shaped!individuals.

Maintains!identities!automatically!in!videos!of!any!length,!enabling!tracking!with!complex!crossings,!occlusions,!or!perturba

tions.!Ability!to!identify!individuals!

across!videos.

idSocial!

(R.!Hinz,!unpublished)!

is!an!accompanying!open!source!analysis!pa

ckage!giving!mean!distances,!distribution!of!distances,!kinematic!parameters!of!interactions,!probabilities!for!relative!dist

ances,!

aggression!leadership!or!followership!hierarchies!and!collective!behavior.

Movement!and!collective!behavior!in!fish,!flies,

mice,!and!ants!

[

]

LoliTrack

(

Movie S

)

Availability)and)usability

commercial,!simple!to!use,!flexible

)

Raw)output

2D!trajectories,!simple!pose

)

Requirements

Win

)

Maximum)individuals

)

Shape)requirements

flexible

Identity)method

crosses!solved

)

Analysis)tools

basic!kinematics

)

Taxa)studied

fish,!rodents,!birds,!insects,!crustaceans,!more

)

Required)resolution)per)individual

9!pixels

)

Notes

Developed!by!

Loligo!Systems.!Available!online!at!

http://www.loligosystems.com/?action=shop_show&varenr=AB10190

Free!trial!version!available.!Valid!for!conventional!cameras!and!laboratory!set

ups,!including!

multiple!arenas.!Software!only!or!with!integrated!hardware.!Does!not!require!constant!and!uncluttered!background.!Ability!to!

easily!characterize!spatial!components!of!the!physical!environment!for!integration!with!tracking!

data.!Simple!and!ea

use!interface.!Shuttlesoft!(

http://www.loligosystems.com/?action=shop_show&varenr=AB10202

)!is!a!dedicated!multi

chamber!system!for!analysis!of!preference!or!avoidance!to!en

vironmental!

drivers,!such!as!temperature!or!dissolved!compounds.!

Three!points!along!body!axis.!

No!inherent!contour.!Can!track!wide!range!of!size!and!shaped!individuals.!

Error!rate!data!not!available.

Basic!measurements!per!

individual,!such!as!body!velo

city,!distance!moved,!turning!rate,!time!spent!in!zone.!Dedicated!solutions!avaliable!upon!request.

See!all!publications!at!

http://www.lolig

osystems.com/index.php?action=references_show_all&menu=14&keyword=VIDEO%20TRACKING

MiceProfiler

[

]

(

Movie!S

)

Availability)and)usability

open

source,!simple!to!use,!flexible

)

Raw)output

2D!trajectory,!detailed!pose

)

Requirements

Win/Linux/Mac

)

Maximum)individuals

(note!2)

)

Shape)requirements

rodent

)

Identity)method

crosses!solved

)

Analysis)tools

extensive

)

Taxa)studied

rodents

)

Required)resolution)per)individual

350!pixels!

)

Notes

Available!online!at!

http://icy.bioimageanalysis.org/plugin/Mice_Profiler_Tracker

Easy!to!use!and!adaptable!to!different!laboratory!setups.!Requires!unchanging!and!uncluttered!background.!Tracker!struggles!w

hen!

individuals!are!in!very!close!contact.

Behavioral!analysis!limit

ed!to!two!individuals.!

Developed!for!mice.

Switches!identities!on!average!twice!per!minute.!Includes!tool!to!facilitate!manual!correction.

System!includes!a!

behavioral!chronogram!generator!(e.g.,!contact!events,!sniffing,!chase,!escape)!and!a!temporal!b

ehavioral!analysis!module.

Social!interactions!in!mice!

[

]

,!rats,!guinea!pigs!(unpublished).

Motr

[

]

(

Movie!S

Availability)and)usability

open

source,!simple!to!use,!flexible

)

Raw)output

2D!trajectory,!simple!pose

)

Requirements

Win/Linux/Mac,!Matlab

)

Maximum)individuals

(note!4)

)

Shape)requirements

elliptical

)

Identity)method

maintained!(marking)

)

Analysis)tools

extensive,!see!JAABA

)

Taxa)studied

mice

)

Required)resolution)per)individual

5000!pixels

)

Notes

Available!at!

http://motr.janelia.org

.!Can!use!conventional!cameras.!Requires!unchanging!and!uncluttered!background!and!uniform!illumination.!

Pose!represented!as!an!ellipse,!including!x,y!coordinates,!major!and!

minor

axis!of!ellipse,!and!ellipse!orientation.!

Matlab!2009b!or!later.

System

validated!up!to!6!individuals,!possibly!can!manage!more.!

Developed!for!rodents,!but!generalizable!to!other!elliptical

shaped!organisms.!

Requires!

that!individuals!are!marked!

–

migh

t!also!work!with!natural!differences!in!pattern,!but!this!has!not!been!tested.!Th

e!system!automatically!learns!external!appearance!of!individuals!from!training!videos!and!generalizes!to!

identify!them!in!a!group.

JAABA!

[

]

was!used!to!learn!complex!behaviors!such!as!following!and!chasing!(for!details!on!JAABA!

[

]

see!Ctrax!

note!6).!

Social!development!in!group

s!of!mice!

[

]

their!standard!configuration!

(1024!x!768!pixel!resolution),!they!fit!a!small!rectangle!(50!x!100!pixels)!around!each!mouse!to!extract!features!needed!for!

individual!identification.!With!these!specifications,!each!mouse!body!formed!an!ellipse!with!a!long!a

xis!

~30

85!pixels,!and!the!small!axis!~10

45!pixels.!

Multitrack

(

Movie S

)

Availability)and)usability

open

source,!simple!to!use,!flexible

)

Raw)output

2D!trajectory,!orientation

)

Requirements

Linux,!additional!libraries

)

Maximum)individuals

1000

(note!3)

)

Shape

)

requirements

flexible

)

Identity)method

not!maintained

)

Analysis)tools

none

)

Taxa)studied

ants,!bees

)

Required)resolution)per)individual

3!pixels

)

Notes

Avaliable!online!at!

www.bio

tracking.org/category/software

.!Flexible!to!different!setups.!Can!use!conventional!cameras.!Requires!constant!background.!System!packed!with!companion!prog

rams!for!creating!shape!

models!and!backgrounds!for!different!experimental!setups.

OpenCV!and

PCL!(standard!computer!vision!libraries),!and!Qt!4.0!(for!creating!graphical!user!interfaces).!All!three!are!cross!platform!a

nd!freely!available.!

Maximum!

tested!~1000!individuals,!but!can!track!an!arbitrarily!large!number.!

Individuals!are!assumed!to!be!

rigid!(see!note!1).!Different!types!of!body!shapes!can!be!tracked!simultaneously!by!creating!multiple!models.!

A!proportion!of!

crossings!can!be!solved!with!movement!models.!

Only!produces!tracking!data.!

Spatial!dynamics!of!ant!

[

]

and!bee!(

B.!Hrolenok,!

unpublished)!behavior.!

Can!be!set!as!a!threshold.!

)

Multi

Worm)Tracker

[

]

(

Movie!S

)

Availability)and)usability

open

source,!simple!to!use,!

flexible

)

Raw)output

2D!trajectory,!detailed!pose!(real

time)

)

Requirements

Win,

LabVIEW!Vision!(NI)!run

time!license

)

Maximum)individuals

(note!4)

Shape)requirements

worm

)

Identity)method

not!maintained

)

Analysis)tools

basic!kinematics,!basic!behaviour

)

Taxa)studied

nematodes,!fly!larvae

)

Required)resolution)per)individual

160!pixels

Notes

Available!online!at!

http://sourceforge.net/projects/mwt/

Requires!

unchanging!and!uncluttered!background.!Can!use!conventional!cameras

Includes!position!of!centroid,!body!size!(number!of!pixels),!vectors!defining!

long!(and!orthogonal)!axes!of!the!shape,!skeleton!(11!point!line!along!midline),!and!outline!of!body!(compre

ssed!bitcode!stored!as!a!string).!Summary!file!can!store!when!a!stimuli!occurred,!and!13!simple!statistics!(number!of!

objects,!mean!speed,!mean!size,!etc.).!

Choreography!runs!on!Win/Linux/Mac.!

p!to!~500!individuals!with!lower!frame!rate!(~10

15!fps).

ptimized!for!

C.,elegans

Identities!not!maintained!

–

trajectories!lost!when!animals!

touch.

Choreography!is!an!offline!basic!behavioral!analysis!package!(direction,!velocity).!Plugins!can!compute!organism

or!condition

specific!behaviors.

Determinants

of!behaviour!in!

C.,elegans

[

]

,!neural!biology!of!

Drosophila

larvae!

[

]

PhenoTracker

(

Movie S

)

Availability)and)usability

commercial,!easy!to!use,!flexible

)

Raw)output

2D!trajectories,!basic!pose

)

Requirements

Win

)

Maximum)individuals

(note!4)

)

Shape)requirements

flexible

Identity)method

maintained!(size/markings)

)

Analysis)tools

basic!kinematics,!social!behaviour

)

Taxa)studied

flies,!ants,!fish,!rodents

)

Required)resolution)per)individual

200!pixels

)

Notes

Developed!by!TSE!S

ystems.!Available!online!at!

http://www.tse

systems.com/products/behavior/video

tracking

software/phenotracker/index.htm

Free!trial!version!available.!Software!only!or!with!integ

rated!

hardware.!A!constant!and!uncluttered!background!not!required.!Shadows,!bedding!material!and!objects!can!be!present!in!the!are

na.!Valid!for!conventional!cameras!and!laboratory!setups,!including!multiple!arenas.!Online!

support!avaliable.!

Identity!not!

maintained!if!individuals!not!marked.!Basic!pose!icludes!head,!center!of!body,!and!tail.!

Server!version!also!available,!requiring!web!browser.!

Maximum!tested,!but!in!principle!could!do!more.!

Basic

measurements!per!individual,!such!as!body!velocity,!distance!moved,!turning!rate,!time!spent!in!zone.!Basic!analysis!for!soci

al!interactions,!including!orientation!towards!or!distance!from!other!individuals.!Can!do!

statistical!analyses!between!videos.

e!all!publications!at!http://www.tse

systems.com/cgi

bin/refsearchn.pl?type=pub.!

Requires!4!pixels!per!individual!for!position!only.

SOS

track

[

]

(

Movie!S

)

Availability)and)usability

open

source,!simple!to!use,!flexible

)

Raw)output

2D!trajectory,!detailed!pose!(real

time)

)

Requirements

Win/Linux/Mac,!Matlab

)

Maximum)individuals

(note!3)

)

Shape)requirements

flexible

Identity)method

not!maintained

)

Analysis)tools

basic!kinematics,!sensory!biology

)

Taxa)studied

flatworm,!larvae,!fly,!fish,!rodent

)

Required)resolution)per)individual

variable

)

Notes

Available!online!at!

http://sourceforge.net/projects/sos

track/

Valid!for!conventional!cameras!and!laboratory!setups.!Suited!for!different!organisms!and!arenas.!Can!track!several!arenas!sim

ultaneously.!Ability!to!add!

and!track!accurate!information!about!the!environment

organism!interaction.!Ability!to!easily!correct!er

rors!(e.g.,!head

tail!swaps).!Requires!constant!uncluttered!background.!

Includes!position!of!centroid,!head,!tail,!and!

midpoint,!and!skeleton,!curvature,!and!total!area.!Real

time!data!does!not!include!pose!information.!

Main!package!can!track!a!single!in

dividual.!Extendable!to!multiple!individuals!(

Movie!S

),!but!loses!identity!when!

individuals!are!similar!(code!available!from!authors!upon!request).

Wide!range!of!size!and!shaped!individuals!can!be!tracked,!including!elliptical,!worm

shaped,!and!legged!organisms.

see!note!3.!

Low

level!motorsensory!

measurements!(body!posture,!kinematic!variables!

–

angles,!velocities,!distances)!and!sensory!information

relevant!to!the!individual!(i.e.,!system!can!map!relevant!points!along!the!individuals!body!to!the!sensory!landscape,!

such!as!local!orientation!relative!to!the!sensory!gradient,!or!stimulus!intensity!at!the!front!of!the!animal).!Automated!basi

c!behavior!c

lassification!available!for!worm

shaped!animals:!turns!and!head!casts.!

Environmental!

control!of!behavior!in!

Drosophila

larva!

[

]

,!other!taxa!(unpublishe

d).!

Can!be!set!as!a!threshold,!although!pose!tracking!impossible!at!low!resolutions.!

ZebraZoom

[

]

(

Movie S

)

Availability)and)usability

open

source,!simple!to!use,!specific

)

Raw)output

2D!trajectory,!basic!pose

)

Requirements

Linux,!Matlab,!

OpenCV,!C++

)

Maximum)individuals

(note!3)

)

Shape)requirements

zebrafish

Identity)method

crosses!solved

)

Analysis)tools

basic!kinematics,!basic!behaviour

)

Taxa)studied

zebrafish

)

Required)resolution)per)individual

350!pixels

)

Notes

Free

online!at!

http://sourceforge.net/p/zebrazoom/wiki/Home/

Limited!flexibility!to!other!setups.!

High

speed!camera!(~330!Hz).!Probably!requires!tuning!of!several!parameters!to!adapt!it!to!a!new!se

up.!

Not!ideal!

for!interactions!involving!close!encounters!between!individuals.

Tested!on!Linux.

Validation!for!up!to!7!individuals,!possibly!system!can!deal!with!more.!

Optimized

for!zebrafish!larvae,!probably!not!generalizable!to!many!

other!fish!speci

es.

Tracking!algorithm!not!robust!to!crossings!

switching!of!identification!between!two!larvae!estimated!to!occur!every!~109!s!(density!of!7!individuals).!

Solves!crossings,!making!on!average!one!mistake!per!

animal!every!110!seconds.

For!each!behavior!de

tected,!extraction!of!output!parameters!per!swim!burst!(i.e,!number!of!oscillations,!tail!beat!frequency,!duration,!orientati

on,!distance!travelled,!mean!speed)!and!parameters!

also!calculated!for!population.!

Attached!behavioral!cluster!package!includes!an!

automatic!(trained)!classifier!of!behaviors!in!three!classes:!slow!forward!swim,!routine!turn,!and!scape.

Genetic!and!environmental!drivers!of!

behavior!in!zebrafish!

[

]

References

1 Matsumoto, J.

, et al.

(2013) A 3D

Video

Based Computerized Analysis of Social and Sexual Interactions in Rats.

PLoS ONE

8, e78460

2 Dankert, H.

, et

al.

(2009) Automated monitoring and analysis of social behavior in Drosophila.

Nat Methods

6, 297

303

3 Wang, L. and Anderson, D.J. (2010) Identification of an aggression

promoting pheromone and its receptor neurons in Drosophila.

Nature

463, 227

231

4 Bra

nson, K.

, et al.

(2009) High

throughput ethomics in large groups of Drosophila.

Nat Methods

6, 451

457

5 Kabra, M.

, et al.

(2013) JAABA: interactive machine learning for automatic annotation of animal behavior.

Nat Methods

10, 64

6 Ofstad, T.A.

, et al.

(2011) Visual place learning in Drosophila melanogaster.

Nature

474, 204

207

7 Bender, J.A.

, et al.

(2011) Kinematic and behavioral evidence for a distinction between trotting and ambling gaits in the cockroach Blaberus disco

idalis.

J Exp Biol

214, 2057

8 Herbert

Read, J.E.

, et al.

(2011) Inferring the rules of interaction of shoaling fish.

Proc Natl Acad Sci U S A

108, 18726

18731

9 Straw, A.D.

, et al.

(2011) Multi

camera real

time three

dimensional tracking of multiple flying animals.

J R Soc

Interface

8, 395

409

10 Maimon, G.

, et al.

(2008) A simple vision

based algorithm for decision making in flying Drosophila.

Curr Biol

18, 464

470

11 van Breugel, F. and Dickinson, M.H. (2012) The visual control of landing and obstacle avoidance in the frui

t fly Drosophila melanogaster.

J Exp Biol

215, 1783

1798

12 Straw, A.D.

, et al.

(2010) Visual control of altitude in flying Drosophila.

Curr Biol

20, 1550

1556

13 Censi, A.

, et al.

(2013) Discriminating external and internal causes for heading changes in freely flying Drosophila.

PLoS Comput Biol

9, e1002891

14 Pérez

Escudero, A.

, et al.

(2014) idTracker: Tracking individuals in a group by automatic identification of unmarked animal

Nature Methods

DOI: 10.1038/nmeth.2994

15 de Chaumont, F.

, et al.

(2012) Computerized video analysis of social interactions in mice.

Nat Methods

9, 410

417

16 Weissbrod, A.

, et al.

(2013) Automated long

term tracking and social behavioural phenotyping

of animal colonies within a semi

natural environment.

Nat Commun

4, 2018

17 Ohayon, S.

, et al.

(2013) Automated multi

day tracking of marked mice for the analysis of social behaviour.

J Neurosci Methods

219, 10

18 Hrolenok, B. and Balch, T. (2013) Learn

ing Executable Models of Multiagent Behavior from Live Animal Observation.

ICML 2013 Workshop on Machine Learning For System Indentification

19 Swierczek, N.A.

, et al.

(2011) High

throughput behavioral analysis in C. elegans.

Nat Methods

8, 592

598

20 Piz

zo, A.B.

, et al.

(2013) The membrane raft protein Flotillin

1 is essential in dopamine neurons for amphetamine

induced behavior in Drosophila.

Mol Psychiatry

18, 824

833

21 Wu, Z.

, et al.

(2011) A combinatorial semaphorin code instructs the initial steps o

f sensory circuit assembly in the Drosophila CNS.

Neuron

70, 281

298

22 Gomez

Marin, A.

, et al.

(2012) Automated tracking of animal posture and movement during exploration and sensory orientation behaviors.

PLoS ONE

7, e41642

23 Gomez

Marin, A.

, et al.

(20

11) Active sampling and decision making in Drosophila chemotaxis.

Nat Commun

2, 441

24 Mirat, O.

, et al.

(2013) ZebraZoom: an automated program for high

throughput behavioral analysis and categorization.

Front Neural Circuits

7, 107

Table S

Criteria for describing the

automated

imag

based

tracking

systems in

Table S1