Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published December 2016 | Published + Supplemental Material + Submitted
Book Section - Chapter Open

Generating Long-term Trajectories Using Deep Hierarchical Networks


We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations. For instance, in sports, agents often choose action sequences with long-term goals in mind, such as achieving a certain strategic position. Conventional policy learning approaches, such as those based on Markov decision processes, generally fail at learning cohesive long-term behavior in such high-dimensional state spaces, and are only effective when fairly myopic decision-making yields the desired behavior. The key difficulty is that conventional models are "single-scale" and only learn a single state-action policy. We instead propose a hierarchical policy class that automatically reasons about both long-term and short-term goals, which we instantiate as a hierarchical neural network. We showcase our approach in a case study on learning to imitate demonstrated basketball trajectories, and show that it generates significantly more realistic trajectories compared to non-hierarchical baselines as judged by professional sports analysts.

Additional Information

© 2016 Neural Information Processing Systems Foundation, Inc. This research was supported in part by NSF Award #1564330, and a GPU donation (Tesla K40 and Titan X) by NVIDIA.

Attached Files

Submitted - 1706.07138.pdf

Supplemental Material - 6520-generating-long-term-trajectories-using-deep-hierarchical-networks-supplemental.zip

Published - 6520-generating-long-term-trajectories-using-deep-hierarchical-networks.pdf



Additional details

August 19, 2023
August 19, 2023