CaltechAUTHORS
  A Caltech Library Service

Entropy-Regularized Stochastic Games

Savas, Yagiz and Ahmadi, Mohamadreza and Tanaka, Takashi and Topcu, Ufuk (2019) Entropy-Regularized Stochastic Games. In: 2019 IEEE 58th Conference on Decision and Control (CDC). IEEE , Piscataway, NJ, pp. 5955-5962. ISBN 9781728113982. https://resolver.caltech.edu/CaltechAUTHORS:20200911-133139267

[img] PDF - Submitted Version
See Usage Policy.

358kB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200911-133139267

Abstract

In zero-sum stochastic games, where two competing players make decisions under uncertainty, a pair of optimal strategies is traditionally described by Nash equilibrium and computed under the assumption that the players have perfect information about the stochastic transition model of the environment. However, implementing such strategies may make the players vulnerable to unforeseen changes in the environment. In this paper, we introduce entropy-regularized stochastic games where each player aims to maximize the causal entropy of its strategy in addition to its expected payoff. The regularization term balances each player's rationality with its belief about the level of misinformation about the transition model. We consider both entropy-regularized N-stage and entropy-regularized discounted stochastic games, and establish the existence of a value in both games. Moreover, we prove the sufficiency of Markovian and stationary mixed strategies to attain the value, respectively, in N-stage and discounted games. Finally, we present algorithms, which are based on convex optimization problems, to compute the optimal strategies. In a numerical example, we demonstrate the proposed method on a motion planning scenario and illustrate the effect of the regularization term on the expected payoff.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
https://doi.org/10.1109/cdc40024.2019.9029555DOIArticle
https://arxiv.org/abs/1907.11543arXivDiscussion Paper
ORCID:
AuthorORCID
Ahmadi, Mohamadreza0000-0003-1447-3012
Additional Information:© 2019 IEEE. This work was supported in part by the grants AFRL # FA9550-19-1-0169 and DARPA # D19AP00004.
Group:Center for Autonomous Systems and Technologies (CAST)
Funders:
Funding AgencyGrant Number
Air Force Research Laboratory (AFRL)FA9550-19-1-0169
Defense Advanced Research Projects Agency (DARPA)D19AP00004
DOI:10.1109/cdc40024.2019.9029555
Record Number:CaltechAUTHORS:20200911-133139267
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200911-133139267
Official Citation:Y. Savas, M. Ahmadi, T. Tanaka and U. Topcu, "Entropy-Regularized Stochastic Games," 2019 IEEE 58th Conference on Decision and Control (CDC), Nice, France, 2019, pp. 5955-5962, doi: 10.1109/CDC40024.2019.9029555
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:105357
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:11 Sep 2020 22:07
Last Modified:16 Nov 2021 18:42

Repository Staff Only: item control page