CaltechAUTHORS
  A Caltech Library Service

Robust Control of Uncertain Markov Decision Processes with Temporal Logic Specifications

Wolff, Eric M. and Topcu, Ufuk and Murray, Richard M. (2011) Robust Control of Uncertain Markov Decision Processes with Temporal Logic Specifications. California Institute of Technology , USA. (Unpublished) http://resolver.caltech.edu/CaltechCDSTR:2011.008

[img]
Preview
PDF
See Usage Policy.

338Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechCDSTR:2011.008

Abstract

We present a method for designing robust controllers for dynamical systems with linear temporal logic specifications. We abstract the original system by a finite Markov Decision Process (MDP) that has transition probabilities in a specified uncertainty set. A robust control policy for the MDP is generated that maximizes the worst-case probability of satisfying the specification over all transition probabilities in the uncertainty set. To do this, we use a procedure from probabilistic model checking to combine the system model with an automaton representing the specification. This new MDP is then transformed into an equivalent form that satisfies assumptions for stochastic shortest path dynamic programming. A robust version of dynamic programming allows us to solve for a $\epsilon$-suboptimal robust control policy with time complexity $O(\log 1/\epsilon)$ times that for the non-robust case. We then implement this control policy on the original dynamical system.


Item Type:Report or Paper (Technical Report)
Additional Information:The authors would like to thank Scott Livingston for helpful comments. This work was supported by an NSF Graduate Research Fellowship and the Boeing Corporation.
Group:Control and Dynamical Systems Technical Reports
Subject Keywords:Temporal logic; Markov decision processes; robotics; robust control
Record Number:CaltechCDSTR:2011.008
Persistent URL:http://resolver.caltech.edu/CaltechCDSTR:2011.008
Usage Policy:You are granted permission for individual, educational, research and non-commercial reproduction, distribution, display and performance of this work in any format.
ID Code:28147
Collection:CaltechCDSTR
Deposited By: Imported from CaltechCDSTR
Deposited On:26 Sep 2011
Last Modified:26 Dec 2012 14:31

Repository Staff Only: item control page