CaltechAUTHORS
  A Caltech Library Service

Learning Calibratable Policies using Programmatic Style-Consistency

Zhan, Eric and Tseng, Albert and Yue, Yisong and Swaminathan, Adith and Hausknecht, Matthew (2019) Learning Calibratable Policies using Programmatic Style-Consistency. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20200109-101924329

[img] PDF - Submitted Version
See Usage Policy.

2254Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200109-101924329

Abstract

We study the important and challenging problem of controllable generation of long-term sequential behaviors. Solutions to this problem would impact many applications, such as calibrating behaviors of AI agents in games or predicting player trajectories in sports. In contrast to the well-studied areas of controllable generation of images, text, and speech, there are significant challenges that are unique to or exacerbated by generating long-term behaviors: how should we specify the factors of variation to control, and how can we ensure that the generated temporal behavior faithfully demonstrates diverse styles? In this paper, we leverage large amounts of raw behavioral data to learn policies that can be calibrated to generate a diverse range of behavior styles (e.g., aggressive versus passive play in sports). Inspired by recent work on leveraging programmatic labeling functions, we present a novel framework that combines imitation learning with data programming to learn style-calibratable policies. Our primary technical contribution is a formal notion of style-consistency as a learning objective, and its integration with conventional imitation learning approaches. We evaluate our framework using demonstrations from professional basketball players and agents in the MuJoCo physics environment, and show that our learned policies can be accurately calibrated to generate interesting behavior styles in both domains.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/1910.01179arXivDiscussion Paper
ORCID:
AuthorORCID
Yue, Yisong0000-0001-9127-1989
Swaminathan, Adith0000-0001-9935-6530
Record Number:CaltechAUTHORS:20200109-101924329
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200109-101924329
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:100592
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:09 Jan 2020 19:41
Last Modified:09 Mar 2020 13:19

Repository Staff Only: item control page