A Caltech Library Service

Limits of Probabilistic Safety Guarantees when Considering Human Uncertainty

Cheng, Richard and Murray, Richard M. and Burdick, Joel W. (2021) Limits of Probabilistic Safety Guarantees when Considering Human Uncertainty. In: 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE , Piscataway, NJ, pp. 3182-3189. ISBN 978-1-7281-9077-8.

[img] PDF - Submitted Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


When autonomous robots interact with humans, such as during autonomous driving, explicit safety guarantees are crucial in order to avoid potentially life-threatening accidents. Many data-driven methods have explored learning probabilistic bounds over human agents’ trajectories (i.e. confidence tubes that contain trajectories with probability δ), which can then be used to guarantee safety with probability 1− δ. However, almost all existing works consider δ ≥ 0.001. The purpose of this paper is to argue that (1) in safety-critical applications, it is necessary to provide safety guarantees with δ < 10⁻⁸, and (2) current learning-based methods are illequipped to compute accurate confidence bounds at such low δ. Using human driving data (from the highD dataset), as well as synthetically generated data, we show that current uncertainty models use inaccurate distributional assumptions to describe human behavior and/or require infeasible amounts of data to accurately learn confidence bounds for δ ≤ 10⁻⁸. These two issues result in unreliable confidence bounds, which can have dangerous implications if deployed on safety-critical systems.

Item Type:Book Section
Related URLs:
URLURL TypeDescription Paper
Cheng, Richard0000-0001-8301-9169
Murray, Richard M.0000-0002-5785-7481
Additional Information:© 2021 IEEE.
Record Number:CaltechAUTHORS:20210511-085543440
Persistent URL:
Official Citation:R. Cheng, R. M. Murray and J. W. Burdick, "Limits of Probabilistic Safety Guarantees when Considering Human Uncertainty," 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 3182-3189, doi: 10.1109/ICRA48506.2021.9561843
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:109070
Deposited By: Tony Diaz
Deposited On:11 May 2021 17:24
Last Modified:14 Dec 2021 21:39

Repository Staff Only: item control page