CaltechAUTHORS
  A Caltech Library Service

Control Theory Meets POMDPs: A Hybrid Systems Approach

Ahmadi, Mohamadreza and Jansen, Nils and Wu, Bo and Topcu, Ufuk (2020) Control Theory Meets POMDPs: A Hybrid Systems Approach. IEEE Transactions on Automatic Control . ISSN 0018-9286. (In Press) https://resolver.caltech.edu/CaltechAUTHORS:20201105-145616123

[img]
Preview
PDF - Accepted Version
See Usage Policy.

1MB
[img]
Preview
PDF - Submitted Version
See Usage Policy.

1MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20201105-145616123

Abstract

Partially observable Markov decision processes(POMDPs) provide a modeling framework for a variety of sequential decision making under uncertainty scenarios in artificial intelligence (AI). Since the states are not directly observable ina POMDP, decision making has to be performed based on the output of a Bayesian filter (continuous beliefs); hence, making POMDPs intractable to solve and analyze. To overcome the complexity challenge of POMDPs, we apply techniques from control theory. Our contributions are fourfold: (i) We begin by casting the problem of analyzing a POMDP into analyzing the behavior of a discrete-time switched system. Then, (ii) in order to estimate the reachable belief space of a POMDP, i.e., the set of all possible evolutions given an initial belief distribution over the states and a set of actions and observations, we find over-approximations in terms of sub-level sets of Lyapunov-like functions. Furthermore, (iii) in order to verify safety and performance requirements of a given POMDP, we formulate a barrier certificate theorem.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1109/tac.2020.3035755DOIArticle
https://arxiv.org/abs/1905.08095arXivDiscussion Paper
ORCID:
AuthorORCID
Ahmadi, Mohamadreza0000-0003-1447-3012
Additional Information:© 2020 IEEE. This work was supported by AFOSR FA9550-19-1-0005, AFRL FA9550-19-1-0169, DARPA D19AP00004, NSF 1646522, NWO OCENW.KLEIN.187, and NSF 1652113 grants. M. Ahmadi appreciates the stimulating discussions with Dr. Yuxin Chen at the University of Chicago, Prof. Yisong Yue at Caltech, and Prof. Richard M. Murray at Caltech.
Group:Center for Autonomous Systems and Technologies (CAST)
Funders:
Funding AgencyGrant Number
Air Force Office of Scientific Research (AFOSR)FA9550-19-1-0005
Air Force Research Laboratory (AFRL)FA9550-19-1-0169
Defense Advanced Research Projects Agency (DARPA)D19AP00004
NSFCNS-1646522
Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO)OCENW.KLEIN.187
NSFCNS-1652113
Record Number:CaltechAUTHORS:20201105-145616123
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20201105-145616123
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:106456
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:06 Nov 2020 15:51
Last Modified:06 Nov 2020 15:51

Repository Staff Only: item control page