A Caltech Library Service

Detect-and-Track: Efficient Pose Estimation in Videos

Girdhar, Rohit and Gkioxari, Georgia and Torresani, Lorenzo and Paluri, Manohar and Tran, Du (2018) Detect-and-Track: Efficient Pose Estimation in Videos. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE , Piscataway, NJ, pp. 350-359. ISBN 978-1-5386-6420-9.

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item:


This paper addresses the problem of estimating and tracking human body keypoints in complex, multi-person video. We propose an extremely lightweight yet highly effective approach that builds upon the latest advancements in human detection [17] and video understanding [5]. Our method operates in two-stages: keypoint estimation in frames or short clips, followed by lightweight tracking to generate keypoint predictions linked over the entire video. For frame-level pose estimation we experiment with Mask R-CNN, as well as our own proposed 3D extension of this model, which leverages temporal information over small clips to generate more robust frame predictions. We conduct extensive ablative experiments on the newly released multi-person video pose estimation benchmark, PoseTrack, to validate various design choices of our model. Our approach achieves an accuracy of 55.2% on the validation and 51.8% on the test set using the Multi-Object Tracking Accuracy (MOTA) metric, and achieves state of the art performance on the ICCV 2017 PoseTrack keypoint tracking challenge [1].

Item Type:Book Section
Related URLs:
URLURL TypeDescription ItemDiscussion Paper
Tran, Du0000-0001-9673-7194
Additional Information:Authors would like to thank Deva Ramanan and Ishan Misra for many helpful discussions. This research is based upon work supported in part by NSF Grant 1618903, the Intel Science and Technology Center for Visual Cloud Systems (lSTC-VCS), Google, and the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA R&D Contract No. D17PC00345. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon.
Funding AgencyGrant Number
Intel Science and Technology Center for Visual Cloud SystemsUNSPECIFIED
Office of the Director of National IntelligenceUNSPECIFIED
Intelligence Advanced Research Projects Activity (IARPA)D17PC00345
Record Number:CaltechAUTHORS:20221215-789753000.13
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:118373
Deposited By: George Porter
Deposited On:20 Dec 2022 23:51
Last Modified:20 Dec 2022 23:51

Repository Staff Only: item control page