CaltechAUTHORS
  A Caltech Library Service

Efficient Exploration Through Bayesian Deep Q-Networks

Azizzadenesheli, Kamyar and Brunskill, Emma and Anandkumar, Animashree (2018) Efficient Exploration Through Bayesian Deep Q-Networks. In: 2018 Information Theory and Applications Workshop (ITA). IEEE , Piscataway, NJ, pp. 1-9. ISBN 9781728101248. http://resolver.caltech.edu/CaltechAUTHORS:20181101-121222226

[img] PDF - Submitted Version
See Usage Policy.

2899Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20181101-121222226

Abstract

We propose Bayesian Deep Q-Network (BDQN), a practical Thompson sampling based Reinforcement Learning (RL) Algorithm. Thompson sampling allows for targeted exploration in high dimensions through posterior sampling but is usually computationally expensive. We address this limitation by introducing uncertainty only at the output layer of the network through a Bayesian Linear Regression (BLR) model, which can be trained with fast closed-form updates and its samples can be drawn efficiently through the Gaussian distribution. We apply our method to a wide range of Atari games in Arcade Learning Environments. Since BDQN carries out more efficient exploration, it is able to reach higher rewards substantially faster than a key baseline, double deep Q network DDQN.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://doi.org/10.1109/ita.2018.8503252DOIArticle
https://arxiv.org/abs/1802.04412arXivDiscussion Paper
Additional Information:© 2018 Association for the Advancement of Artificial Intelligence. The authors would like to thank Zachary C. Lipton, Marlos C. Machado, Ian Osband, Gergely Neu, and the anonymous reviewers for their feedback and suggestions.
Subject Keywords:Uncertainty; Games; Bayes methods; Linear regression; Complexity theory; Neural networks
Record Number:CaltechAUTHORS:20181101-121222226
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20181101-121222226
Official Citation:K. Azizzadenesheli, E. Brunskill and A. Anandkumar, "Efficient Exploration Through Bayesian Deep Q-Networks," 2018 Information Theory and Applications Workshop (ITA), San Diego, CA, USA, 2018, pp. 1-9. doi: 10.1109/ITA.2018.8503252
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:90569
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:01 Nov 2018 19:42
Last Modified:27 Mar 2019 15:39

Repository Staff Only: item control page