CaltechAUTHORS
  A Caltech Library Service

On the Utility of Model Learning in HRI

Choudhury, Rohan and Swamy, Gokul and Hadfield-Menell, Dylan and Dragan, Anca D. (2019) On the Utility of Model Learning in HRI. In: 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE , Piscataway, NJ, pp. 317-325. ISBN 9781538685556. https://resolver.caltech.edu/CaltechAUTHORS:20190522-154638585

[img] PDF - Accepted Version
See Usage Policy.

2165Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20190522-154638585

Abstract

Fundamental to robotics is the debate between model-based and model-free learning: should the robot build an explicit model of the world, or learn a policy directly? In the context of HRI, part of the world to be modeled is the human. One option is for the robot to treat the human as a black box and learn a policy for how they act directly. But it can also model the human as an agent, and rely on a “theory of mind” to guide or bias the learning (grey box). We contribute a characterization of the performance of these methods under the optimistic case of having an ideal theory of mind, as well as under different scenarios in which the assumptions behind the robot's theory of mind for the human are wrong, as they inevitably will be in practice. We find that there is a significant sample complexity advantage to theory of mind methods and that they are more robust to covariate shift, but that when enough interaction data is available, black box approaches eventually dominate.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://doi.org/10.1109/hri.2019.8673256DOIArticle
https://arxiv.org/abs/1901.01291arXivDiscussion Paper
Additional Information:© 2019 IEEE. We thank the members of the InterACT Lab at UC Berkeley. In particular, we are grateful for Kush Bhatia's feedback on building human simulators and Eli Bronstein's assistance on the black-box model-based component of this work. This work is partially supported by NVIDIA and the Caltech Arjun Bansal and Ria Langheim Summer Undergraduate Research Fellowship.
Funders:
Funding AgencyGrant Number
nVidiaUNSPECIFIED
Caltech Summer Undergraduate Research Fellowship (SURF)UNSPECIFIED
Subject Keywords:theory of mind, inverse RL, model-based RL, model-free RL, sample complexity
Record Number:CaltechAUTHORS:20190522-154638585
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20190522-154638585
Official Citation:R. Choudhury, G. Swamy, D. Hadfield-Menell and A. D. Dragan, "On the Utility of Model Learning in HRI," 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Daegu, Korea (South), 2019, pp. 317-325. doi: 10.1109/HRI.2019.8673256
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:95719
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:22 May 2019 23:02
Last Modified:03 Oct 2019 21:16

Repository Staff Only: item control page