CaltechAUTHORS
  A Caltech Library Service

Compactly Restrictable Metric Policy Optimization Problems

Dorobantu, Victor D. and Azizzadenesheli, Kamyar and Yue, Yisong (2022) Compactly Restrictable Metric Policy Optimization Problems. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20220714-212414777

[img] PDF - Submitted Version
See Usage Policy.

2MB

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20220714-212414777

Abstract

We study policy optimization problems for deterministic Markov decision processes (MDPs) with metric state and action spaces, which we refer to as Metric Policy Optimization Problems (MPOPs). Our goal is to establish theoretical results on the well-posedness of MPOPs that can characterize practically relevant continuous control systems. To do so, we define a special class of MPOPs called Compactly Restrictable MPOPs (CR-MPOPs), which are flexible enough to capture the complex behavior of robotic systems but specific enough to admit solutions using dynamic programming methods such as value iteration. We show how to arrive at CR-MPOPs using forward-invariance. We further show that our theoretical results on CR-MPOPs can be used to characterize feedback linearizable control affine systems.


Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription
https://doi.org/10.48550/arXiv.2207.05850arXivDiscussion Paper
ORCID:
AuthorORCID
Dorobantu, Victor D.0000-0002-2797-7802
Azizzadenesheli, Kamyar0000-0001-8507-1868
Yue, Yisong0000-0001-9127-1989
Additional Information:Submitted May 15th, 2021. Resubmitted July 6th, 2022. This work was supported in part by DARPA and Beyond Limits. Victor D. Dorobantu was also supported in part by a Kortschak Fellowship.
Funders:
Funding AgencyGrant Number
Defense Advanced Research Projects Agency (DARPA)UNSPECIFIED
Beyond LimitsUNSPECIFIED
Kortschak Scholars ProgramUNSPECIFIED
Subject Keywords:Continuous Markov Decision Processes, Reinforcement Learning, Optimal Control, Value Iteration, Selection Theorems, Sampled-Data, Physical Systems
Record Number:CaltechAUTHORS:20220714-212414777
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20220714-212414777
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:115568
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:15 Jul 2022 22:38
Last Modified:15 Jul 2022 22:38

Repository Staff Only: item control page