CaltechAUTHORS
  A Caltech Library Service

Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting

Lale, Sahin and Azizzadenesheli, Kamyar and Hassibi, Babak and Anandkumar, Anima (2020) Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20200403-141835981

[img] PDF (24 Jun 2020) - Submitted Version
See Usage Policy.

475Kb
[img] PDF (12 Mar 2020) - Submitted Version
See Usage Policy.

462Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200403-141835981

Abstract

We study the problem of adaptive control in partially observable linear quadratic Gaussian control systems, where the model dynamics are unknown a priori. We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty, to effectively minimize the overall control cost. We employ the predictor state evolution representation of the system dynamics and deploy a recently proposed closed-loop system identification method, estimation, and confidence bound construction. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model for further exploration and exploitation. We provide stability guarantees for LqgOpt and prove the regret upper bound of O(√T) for adaptive control of linear quadratic Gaussian (LQG) systems, where T is the time horizon of the problem.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2003.05999arXivDiscussion Paper
ORCID:
AuthorORCID
Azizzadenesheli, Kamyar0000-0001-8507-1868
Alternate Title:Regret Bound of Adaptive Control in Linear Quadratic Gaussian (LQG) Systems
Additional Information:S. Lale is supported in part by DARPA PAI. K. Azizzadenesheli is supported in part by Raytheon and Amazon Web Service. B. Hassibi is supported in part by the National Science Foundation under grants CNS-0932428, CCF-1018927, CCF-1423663 and CCF-1409204, by a grant from Qualcomm Inc., by NASA’s Jet Propulsion Laboratory through the President and Director’s Fund, and by King Abdullah University of Science and Technology. A. Anandkumar is supported in part by Bren endowed chair, DARPA PAIHR00111890035 and LwLL grants, Raytheon, Microsoft, Google, and Adobe faculty fellowships.
Funders:
Funding AgencyGrant Number
Defense Advanced Research Projects Agency (DARPA)PAIHR00111890035
RaytheonUNSPECIFIED
Amazon Web ServicesUNSPECIFIED
NSFCNS-0932428
NSFCCF-1018927
NSFCCF-1423663
NSFCCF-1409204
Qualcomm Inc.UNSPECIFIED
JPL President and Director's FundUNSPECIFIED
King Abdullah University of Science and Technology (KAUST)UNSPECIFIED
Bren Professor of Computing and Mathematical SciencesUNSPECIFIED
Learning with Less Labels (LwLL)UNSPECIFIED
Microsoft Faculty FellowshipUNSPECIFIED
Google Faculty Research AwardUNSPECIFIED
AdobeUNSPECIFIED
Record Number:CaltechAUTHORS:20200403-141835981
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200403-141835981
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:102332
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:03 Apr 2020 21:41
Last Modified:23 Nov 2020 23:08

Repository Staff Only: item control page