CaltechAUTHORS
  A Caltech Library Service

KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems

Lale, Sahin and Shi, Yuanyuan and Qu, Guannan and Azizzadenesheli, Kamyar and Wierman, Adam and Anandkumar, Anima (2022) KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20220714-212504090

[img] PDF - Submitted Version
See Usage Policy.

219kB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20220714-212504090

Abstract

Learning a dynamical system requires stabilizing the unknown dynamics to avoid state blow-ups. However, current reinforcement learning (RL) methods lack stabilization guarantees, which limits their applicability for the control of safety-critical systems. We propose a model-based RL framework with formal stability guarantees, Krasovskii Constrained RL (KCRL), that adopts Krasovskii's family of Lyapunov functions as a stability constraint. The proposed method learns the system dynamics up to a confidence interval using feature representation, e.g. Random Fourier Features. It then solves a constrained policy optimization problem with a stability constraint based on Krasovskii's method using a primal-dual approach to recover a stabilizing policy. We show that KCRL is guaranteed to learn a stabilizing policy in a finite number of interactions with the underlying unknown system. We also derive the sample complexity upper bound for stabilization of unknown nonlinear dynamical systems via the KCRL framework.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
https://doi.org/10.48550/arXiv.2206.01704arXivDiscussion Paper
ORCID:
AuthorORCID
Lale, Sahin0000-0002-7191-346X
Shi, Yuanyuan0000-0002-6182-7664
Qu, Guannan0000-0002-5466-3550
Azizzadenesheli, Kamyar0000-0001-8507-1868
Wierman, Adam0000-0002-5923-0199
Anandkumar, Anima0000-0002-6974-6797
Record Number:CaltechAUTHORS:20220714-212504090
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20220714-212504090
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:115581
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:15 Jul 2022 22:43
Last Modified:23 Dec 2022 00:52

Repository Staff Only: item control page