A Caltech Library Service

Self-Supervised Online Learning for Safety-Critical Control using Stereo Vision

Cosner, Ryan K. and Jimenez Rodriguez, Ivan D. and Molnar, Tamas G. and Ubellacker, Wyatt and Yue, Yisong and Ames, Aaron D. and Bouman, Katherine L. (2022) Self-Supervised Online Learning for Safety-Critical Control using Stereo Vision. . (Unpublished)

[img] PDF - Submitted Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


With the increasing prevalence of complex vision-based sensing methods for use in obstacle identification and state estimation, characterizing environment-dependent measurement errors has become a difficult and essential part of modern robotics. This paper presents a self-supervised learning approach to safety-critical control. In particular, the uncertainty associated with stereo vision is estimated, and adapted online to new visual environments, wherein this estimate is leveraged in a safety-critical controller in a robust fashion. To this end, we propose an algorithm that exploits the structure of stereo-vision to learn an uncertainty estimate without the need for ground-truth data. We then robustify existing Control Barrier Function-based controllers to provide safety in the presence of this uncertainty estimate. We demonstrate the efficacy of our method on a quadrupedal robot in a variety of environments. When not using our method safety is violated. With offline training alone we observe the robot is safe, but overly-conservative. With our online method the quadruped remains safe and conservatism is reduced.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Cosner, Ryan K.0000-0002-4035-1425
Molnar, Tamas G.0000-0002-9379-7121
Ubellacker, Wyatt0000-0002-4732-6185
Yue, Yisong0000-0001-9127-1989
Ames, Aaron D.0000-0003-0848-3177
Bouman, Katherine L.0000-0003-0077-4367
Additional Information:Attribution 4.0 International (CC BY 4.0). This research is supported in part by the National Science Foundation CPS Award #1932091, Dow (#227027AT), BP p.l.c., AeroVironment.
Funding AgencyGrant Number
Dow Chemical Company227027AT
Record Number:CaltechAUTHORS:20220325-220806703
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:114092
Deposited By: George Porter
Deposited On:25 Mar 2022 23:14
Last Modified:25 Mar 2022 23:14

Repository Staff Only: item control page