CaltechAUTHORS
  A Caltech Library Service

Triply Robust Off-Policy Evaluation

Liu, Anqi and Liu, Hao and Anandkumar, Anima and Yue, Yisong (2019) Triply Robust Off-Policy Evaluation. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20200109-085907638

[img] PDF - Submitted Version
See Usage Policy.

630Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200109-085907638

Abstract

We propose a robust regression approach to off-policy evaluation (OPE) for contextual bandits. We frame OPE as a covariate-shift problem and leverage modern robust regression tools. Ours is a general approach that can be used to augment any existing OPE method that utilizes the direct method. When augmenting doubly robust methods, we call the resulting method Triply Robust. We prove upper bounds on the resulting bias and variance, as well as derive novel minimax bounds based on robust minimax analysis for covariate shift. Our robust regression method is compatible with deep learning, and is thus applicable to complex OPE settings that require powerful function approximators. Finally, we demonstrate superior empirical performance across the standard OPE benchmarks, especially in the case where the logging policy is unknown and must be estimated from data.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/1911.05811arXivDiscussion Paper
ORCID:
AuthorORCID
Yue, Yisong0000-0001-9127-1989
Additional Information:Prof. Anandkumar is supported by Bren endowed Chair, faculty awards from Microsoft, Google, and Adobe, DARPA PAI and LwLL grants. Anqi Liu is a PIMCO postdoctoral fellow at Caltech.
Funders:
Funding AgencyGrant Number
Bren Professor of Computing and Mathematical SciencesUNSPECIFIED
MicrosoftUNSPECIFIED
GoogleUNSPECIFIED
AdobeUNSPECIFIED
Defense Advanced Research Projects Agency (DARPA)UNSPECIFIED
Caltech PIMCO Graduate FellowshipUNSPECIFIED
Record Number:CaltechAUTHORS:20200109-085907638
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200109-085907638
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:100578
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:09 Jan 2020 19:46
Last Modified:09 Jan 2020 19:46

Repository Staff Only: item control page