CaltechAUTHORS
  A Caltech Library Service

Explore More and Improve Regret in Linear Quadratic Regulators

Lale, Sahin and Azizzadenesheli, Kamyar and Hassibi, Babak and Anandkumar, Anima (2020) Explore More and Improve Regret in Linear Quadratic Regulators. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20201106-120155157

[img] PDF - Submitted Version
See Usage Policy.

875Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20201106-120155157

Abstract

Stabilizing the unknown dynamics of a control system and minimizing regret in control of an unknown system are among the main goals in control theory and reinforcement learning. In this work, we pursue both these goals for adaptive control of linear quadratic regulators (LQR). Prior works accomplish either one of these goals at the cost of the other one. The algorithms that are guaranteed to find a stabilizing controller suffer from high regret, whereas algorithms that focus on achieving low regret assume the presence of a stabilizing controller at the early stages of agent-environment interaction. In the absence of such a stabilizing controller, at the early stages, the lack of reasonable model estimates needed for (i) strategic exploration and (ii) design of controllers that stabilize the system, results in regret that scales exponentially in the problem dimensions. We propose a framework for adaptive control that exploits the characteristics of linear dynamical systems and deploys additional exploration in the early stages of agent-environment interaction to guarantee sooner design of stabilizing controllers. We show that for the classes of controllable and stabilizable LQRs, where the latter is a generalization of prior work, these methods achieve O(√T) regret with a polynomial dependence in the problem dimensions.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2007.12291arXivDiscussion Paper
ORCID:
AuthorORCID
Azizzadenesheli, Kamyar0000-0001-8507-1868
Record Number:CaltechAUTHORS:20201106-120155157
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20201106-120155157
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:106484
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:06 Nov 2020 21:51
Last Modified:06 Nov 2020 21:51

Repository Staff Only: item control page