A Caltech Library Service

Robust control of uncertain Markov Decision Processes with temporal logic specifications

Wolff, Eric M. and Topcu, Ufuk and Murray, Richard M. (2012) Robust control of uncertain Markov Decision Processes with temporal logic specifications. In: 51st IEEE Conference on Decision and Control. IEEE , Piscataway, NJ, pp. 3372-3379. ISBN 978-1-4673-2065-8.

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item:


We present a method for designing a robust control policy for an uncertain system subject to temporal logic specifications. The system is modeled as a finite Markov Decision Process (MDP) whose transition probabilities are not exactly known but are known to belong to a given uncertainty set. A robust control policy is generated for the MDP that maximizes the worst-case probability of satisfying the specification over all transition probabilities in this uncertainty set. To this end, we use a procedure from probabilistic model checking to combine the system model with an automaton representing the specification. This new MDP is then transformed into an equivalent form that satisfies assumptions for stochastic shortest path dynamic programming. A robust version of dynamic programming solves for a ε-suboptimal robust control policy with time complexity O(log1/ε) times that for the non-robust case.

Item Type:Book Section
Related URLs:
URLURL TypeDescription DOIArticle ItemTechnical Report
Murray, Richard M.0000-0002-5785-7481
Additional Information:© 2012 IEEE. The authors would like to thank Scott Livingston and the anonymous reviewers for helpful comments. This work was supported by an NSF Graduate Research Fellowship, the Boeing Corporation, and AFOSR award FA9550-12-1-0302.
Funding AgencyGrant Number
NSF Graduate Research FellowshipUNSPECIFIED
Boeing CorporationUNSPECIFIED
Air Force Office of Scientific Research (AFOSR)FA9550-12-1-0302
Record Number:CaltechAUTHORS:20131220-132617887
Persistent URL:
Official Citation:Wolff, E.M.; Topcu, U.; Murray, R.M., "Robust control of uncertain Markov Decision Processes with temporal logic specifications," Decision and Control (CDC), 2012 IEEE 51st Annual Conference on , vol., no., pp.3372,3379, 10-13 Dec. 2012 doi: 10.1109/CDC.2012.6426174 URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:43119
Deposited By: Tony Diaz
Deposited On:23 Dec 2013 19:44
Last Modified:10 Nov 2021 16:33

Repository Staff Only: item control page