CaltechAUTHORS
  A Caltech Library Service

Learning recurrent representations for hierarchical behavior modeling

Eyjolfsdottir, Eyrun and Branson, Kristin and Yue, Yisong and Perona, Pietro (2017) Learning recurrent representations for hierarchical behavior modeling. In: 5th International Conference on Learning Representations (ICLR), 24-26 April 2017, Toulon, France. https://resolver.caltech.edu/CaltechAUTHORS:20170530-090151819

[img] PDF - Submitted Version
See Usage Policy.

7Mb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20170530-090151819

Abstract

We propose a framework for detecting action patterns from motion sequences and modeling the sensory-motor relationship of animals, using a generative recurrent neural network. The network has a discriminative part (classifying actions) and a generative part (predicting motion), whose recurrent cells are laterally connected, allowing higher levels of the network to represent high level behavioral phenomena. We test our framework on two types of data, fruit fly behavior and online handwriting. Our results show that 1) taking advantage of unlabeled sequences, by predicting future motion, significantly improves action detection performance when training labels are scarce, 2) the network learns to represent high level phenomena such as writer identity and fly gender, without supervision, and 3) simulated motion trajectories, generated by treating motion prediction as input to the network, look realistic and may be used to qualitatively evaluate whether the model has learnt generative control rules.


Item Type:Conference or Workshop Item (Paper)
Related URLs:
URLURL TypeDescription
https://arxiv.org/abs/1611.00094arXivDiscussion Paper
ORCID:
AuthorORCID
Branson, Kristin0000-0002-5567-2512
Yue, Yisong0000-0001-9127-1989
Perona, Pietro0000-0002-7583-5809
Record Number:CaltechAUTHORS:20170530-090151819
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20170530-090151819
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:77821
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:30 May 2017 17:16
Last Modified:09 Mar 2020 13:19

Repository Staff Only: item control page