A Caltech Library Service

Efficient Exploration Through Bayesian Deep Q-Networks

Azizzadenesheli, Kamyar and Brunskill, Emma and Anandkumar, Animashree (2018) Efficient Exploration Through Bayesian Deep Q-Networks. In: 2018 Information Theory and Applications Workshop (ITA). IEEE , Piscataway, NJ, pp. 1-9. ISBN 9781728101248.

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


We propose Bayesian Deep Q-Network (BDQN), a practical Thompson sampling based Reinforcement Learning (RL) Algorithm. Thompson sampling allows for targeted exploration in high dimensions through posterior sampling but is usually computationally expensive. We address this limitation by introducing uncertainty only at the output layer of the network through a Bayesian Linear Regression (BLR) model, which can be trained with fast closed-form updates and its samples can be drawn efficiently through the Gaussian distribution. We apply our method to a wide range of Atari games in Arcade Learning Environments. Since BDQN carries out more efficient exploration, it is able to reach higher rewards substantially faster than a key baseline, double deep Q network DDQN.

Item Type:Book Section
Related URLs:
URLURL TypeDescription Paper
Azizzadenesheli, Kamyar0000-0001-8507-1868
Brunskill, Emma0000-0002-3971-7127
Anandkumar, Animashree0000-0002-6974-6797
Additional Information:© 2018 Association for the Advancement of Artificial Intelligence. The authors would like to thank Zachary C. Lipton, Marlos C. Machado, Ian Osband, Gergely Neu, and the anonymous reviewers for their feedback and suggestions.
Subject Keywords:Uncertainty; Games; Bayes methods; Linear regression; Complexity theory; Neural networks
Record Number:CaltechAUTHORS:20181101-121222226
Persistent URL:
Official Citation:K. Azizzadenesheli, E. Brunskill and A. Anandkumar, "Efficient Exploration Through Bayesian Deep Q-Networks," 2018 Information Theory and Applications Workshop (ITA), San Diego, CA, USA, 2018, pp. 1-9. doi: 10.1109/ITA.2018.8503252
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:90569
Deposited By: George Porter
Deposited On:01 Nov 2018 19:42
Last Modified:23 Dec 2022 19:40

Repository Staff Only: item control page