CaltechAUTHORS
  A Caltech Library Service

Linearly Solvable Stochastic Control Lyapunov Functions

Leong, Yoke Peng and Horowitz, Matanya B. and Burdick, Joel W. (2016) Linearly Solvable Stochastic Control Lyapunov Functions. SIAM Journal on Control and Optimization, 54 (6). pp. 3106-3125. ISSN 0363-0129. https://resolver.caltech.edu/CaltechAUTHORS:20170203-081725763

[img] PDF - Published Version
See Usage Policy.

2099Kb
[img] PDF - Submitted Version
See Usage Policy.

2288Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20170203-081725763

Abstract

This paper presents a new method for synthesizing stochastic control Lyapunov functions for a class of nonlinear stochastic control systems. The technique relies on a transformation of the classical nonlinear Hamilton--Jacobi--Bellman partial differential equation to a linear partial differential equation for a class of problems with a particular constraint on the stochastic forcing. This linear partial differential equation can then be relaxed to a linear differential inclusion, allowing for relaxed solutions to be generated using sum of squares programming. The resulting relaxed solutions are in fact viscosity super-/subsolutions, and by the maximum principle are pointwise upper and lower bounds to the underlying value function, even for coarse polynomial approximations. Furthermore, the pointwise upper bound is shown to be a stochastic control Lyapunov function, yielding a method for generating nonlinear controllers with pointwise bounded distance from the optimal cost when using the optimal controller. These approximate solutions may be computed with nonincreasing error via a hierarchy of semidefinite optimization problems. Finally, this paper develops a priori bounds on trajectory suboptimality when using these approximate value functions and demonstrates that these methods, and bounds, can be applied to a more general class of nonlinear systems not obeying the constraint on stochastic forcing. Simulated examples illustrate the methodology.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1137/16M105767XDOIArticle
http://epubs.siam.org/doi/10.1137/16M105767XPublisherArticle
https://arxiv.org/abs/1410.0405arXivDiscussion Paper
Additional Information:© 2016 Society for Industrial and Applied Mathematics. Received by the editors February 1, 2016; accepted for publication (in revised form) August 31, 2016; published electronically December 6, 2016. A short version of this work appeared as [24].
Subject Keywords:stochastic control Lyapunov function, sum of squares programming, Hamilton–Jacobi–Bellman equation, nonlinear systems, optimal control
Issue or Number:6
Classification Code:AMS subject classifications: 93E15, 93E20
Record Number:CaltechAUTHORS:20170203-081725763
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20170203-081725763
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:74003
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:03 Feb 2017 16:55
Last Modified:03 Oct 2019 16:33

Repository Staff Only: item control page