CaltechAUTHORS
  A Caltech Library Service

Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems

Lale, Sahin and Azizzadenesheli, Kamyar and Hassibi, Babak and Anandkumar, Anima (2020) Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20200402-131032742

[img] PDF (24 Jun 2020) - Submitted Version
See Usage Policy.

1078Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200402-131032742

Abstract

We study the problem of adaptive control in partially observable linear dynamical systems. We propose a novel algorithm, adaptive control online learning algorithm (AdaptOn), which efficiently explores the environment, estimates the system dynamics episodically and exploits these estimates to design effective controllers to minimize the cumulative costs. Through interaction with the environment, AdaptOn deploys online convex optimization to optimize the controller while simultaneously learning the system dynamics to improve the accuracy of controller updates. We show that when the cost functions are strongly convex, after T times step of agent-environment interaction, AdaptOn achieves regret upper bound of polylog(T). To the best of our knowledge, AdaptOn is the first algorithm which achieves polylog(T) regret in adaptive control of unknown partially observable linear dynamical systems which includes linear quadratic Gaussian (LQG) control.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
http://arxiv.org/abs/2003.11227arXivDiscussion Paper
ORCID:
AuthorORCID
Azizzadenesheli, Kamyar0000-0001-8507-1868
Additional Information:S. Lale is supported in part by DARPA PAI. K. Azizzadenesheli gratefully acknowledge the financial support of Raytheon and Amazon Web Services. B. Hassibi is supported in part by the National Science Foundation under grants CNS-0932428, CCF-1018927, CCF-1423663 and CCF-1409204, by a grant from Qualcomm Inc., by NASA’s Jet Propulsion Laboratory through the President and Director’s Fund, and by King Abdullah University of Science and Technology. A. Anandkumar is supported in part by Bren endowed chair, DARPA PAIHR00111890035 and LwLL grants, Raytheon, Microsoft, Google, and Adobe faculty fellowships.
Funders:
Funding AgencyGrant Number
Defense Advanced Research Projects Agency (DARPA)PAIHR00111890035
RaytheonUNSPECIFIED
Amazon Web ServicesUNSPECIFIED
NSFCNS-0932428
NSFCCF-1018927
NSFCCF-1423663
NSFCCF-1409204
Qualcomm Inc.UNSPECIFIED
JPL President and Director's FundUNSPECIFIED
King Abdullah University of Science and Technology (KAUST)UNSPECIFIED
Bren Professor of Computing and Mathematical SciencesUNSPECIFIED
Learning with Less Labels (LwLL)UNSPECIFIED
Microsoft Faculty FellowshipUNSPECIFIED
Google Faculty Research AwardUNSPECIFIED
AdobeUNSPECIFIED
Record Number:CaltechAUTHORS:20200402-131032742
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200402-131032742
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:102270
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:02 Apr 2020 20:36
Last Modified:06 Nov 2020 22:05

Repository Staff Only: item control page