CaltechAUTHORS
  A Caltech Library Service

Correlational Dueling Bandits with Application to Clinical Treatment in Large Decision Spaces

Sui, Yanan and Yue, Yisong and Burdick, Joel W. (2017) Correlational Dueling Bandits with Application to Clinical Treatment in Large Decision Spaces. . (Submitted) http://resolver.caltech.edu/CaltechAUTHORS:20190205-133559444

[img] PDF - Submitted Version
See Usage Policy.

633Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20190205-133559444

Abstract

We consider sequential decision making under uncertainty, where the goal is to optimize over a large decision space using noisy comparative feedback. This problem can be formulated as a K-armed Dueling Bandits problem where K is the total number of decisions. When K is very large, existing dueling bandits algorithms suffer huge cumulative regret before converging on the optimal arm. This paper studies the dueling bandits problem with a large number of arms that exhibit a low-dimensional correlation structure. Our problem is motivated by a clinical decision making process in large decision space. We propose an efficient algorithm CorrDuel which optimizes the exploration/exploitation tradeoff in this large decision space of clinical treatments. More broadly, our approach can be applied to other sequential decision problems with large and structured decision spaces. We derive regret bounds, and evaluate performance in simulation experiments as well as on a live clinical trial of therapeutic spinal cord stimulation. To our knowledge, this marks the first time an online learning algorithm was applied towards spinal cord injury treatments. Our experimental results show the effectiveness and efficiency of our approach.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/1707.02375arXivDiscussion Paper
Additional Information:This research was supported in part by Caltech/JPL PDF IAMS100224, NIH-U01-EB007615-08, NIH-U01-EB015521-05, and a gift from Northrop Grumman.
Funders:
Funding AgencyGrant Number
JPL President and Director's FundIAMS100224
NIHU01-EB007615-08
NIHU01-EB015521-05
Northrop GrummanUNSPECIFIED
Record Number:CaltechAUTHORS:20190205-133559444
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20190205-133559444
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:92675
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:05 Feb 2019 21:46
Last Modified:05 Feb 2019 21:46

Repository Staff Only: item control page