CaltechAUTHORS
  A Caltech Library Service

Distributed Reinforcement Learning in Multi-Agent Networked Systems

Lin, Yiheng and Qu, Guannan and Huang, Longbo and Wierman, Adam (2020) Distributed Reinforcement Learning in Multi-Agent Networked Systems. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20201014-143549786

[img] PDF - Submitted Version
See Usage Policy.

748Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20201014-143549786

Abstract

We study distributed reinforcement learning (RL) for a network of agents. The objective is to find localized policies that maximize the (discounted) global reward. In general, scalability is a challenge in this setting because the size of the global state/action space can be exponential in the number of agents. Scalable algorithms are only known in cases where dependencies are local, e.g., between neighbors. In this work, we propose a Scalable Actor Critic framework that applies in settings where the dependencies are non-local and provide a finite-time error bound that shows how the convergence rate depends on the depth of the dependencies in the network. Additionally, as a byproduct of our analysis, we obtain novel finite-time convergence results for a general stochastic approximation scheme and for temporal difference learning with state aggregation that apply beyond the setting of RL in networked systems.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2006.06555arXivDiscussion Paper
ORCID:
AuthorORCID
Qu, Guannan0000-0002-5466-3550
Huang, Longbo0000-0002-7341-447X
Additional Information:We see no ethical concerns related to this paper.
Record Number:CaltechAUTHORS:20201014-143549786
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20201014-143549786
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:106067
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:14 Oct 2020 21:43
Last Modified:14 Oct 2020 21:43

Repository Staff Only: item control page