CaltechAUTHORS
  A Caltech Library Service

Coach-Player Multi-agent Reinforcement Learning for Dynamic Team Composition

Liu, Bo and Liu, Qiang and Stone, Peter and Garg, Animesh and Zhu, Yuke and Anandkumar, Animashree (2021) Coach-Player Multi-agent Reinforcement Learning for Dynamic Team Composition. Proceedings of Machine Learning Research, 139 . pp. 6860-6870. ISSN 2640-3498. https://resolver.caltech.edu/CaltechAUTHORS:20210831-203857558

[img] PDF - Published Version
See Usage Policy.

2MB
[img] PDF - Accepted Version
Creative Commons Attribution.

2MB
[img] PDF - Supplemental Material
See Usage Policy.

216kB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20210831-203857558

Abstract

In real-world multi-agent systems, agents with different capabilities may join or leave without altering the team’s overarching goals. Coordinating teams with such dynamic composition is challenging: the optimal team strategy varies with the composition. We propose COPA, a coach-player framework to tackle this problem. We assume the coach has a global view of the environment and coordinates the players, who only have partial views, by distributing individual strategies. Specifically, we 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players. We validate our methods on a resource collection task, a rescue game, and the StarCraft micromanagement tasks. We demonstrate zero-shot generalization to new team compositions. Our method achieves comparable or better performance than the setting where all players have a full view of the environment. Moreover, we see that the performance remains high even when the coach communicates as little as 13% of the time using the adaptive communication strategy.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://proceedings.mlr.press/v139/liu21m.htmlPublisherArticle
http://arxiv.org/abs/2105.08692arXivDiscussion Paper
ORCID:
AuthorORCID
Garg, Animesh0000-0003-0482-4296
Zhu, Yuke0000-0002-9198-2227
Additional Information:© 2021 The authors.
Record Number:CaltechAUTHORS:20210831-203857558
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20210831-203857558
Official Citation:Coach-Player Multi-agent Reinforcement Learning for Dynamic Team Composition Bo Liu, Qiang Liu, Peter Stone, Animesh Garg, Yuke Zhu, Anima Anandkumar Proceedings of the 38th International Conference on Machine Learning, PMLR 139:6860-6870, 2021.
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:110645
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:01 Sep 2021 14:49
Last Modified:01 Sep 2021 14:49

Repository Staff Only: item control page