A Caltech Library Service

Systems and methods for labeling source data using confidence labels

Welinder, Peter and Perona, Pietro (2017) Systems and methods for labeling source data using confidence labels. Patent number: US9704106B2.

[img] PDF (US9704106B2) - Published Version
See Usage Policy.


Use this Persistent URL to link to this item:


Systems and methods for the annotation of source data using confidence labels in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining confidence labels for crowdsourced annotations includes obtaining a set of source data, obtaining a set of training data representative of the set of source data, determining the ground truth for each piece of training data, obtaining a set of training data annotations including a confidence label, measuring annotator accuracy data for at least one piece of training data, and automatically generating a set of confidence labels for the set of unlabeled data based on the measured annotator accuracy data and the set of annotator labels used.

Item Type:Patent
Related Patents:US20130346356A1
Patent Classification:Current U.S. Class: 1/1. Current CPC Class: G06F 17/30598 (20130101); G06F 17/30525 (20130101); G06F 17/30303 (20130101); G06N 5/048 (20130101); G06N 99/005 (20130101); G06F 17/241 (20130101). Current International Class: G06N 5/04 (20060101); G06N 99/00 (20100101); G06F 17/24 (20060101); G06F 17/30 (20060101).
Patent Assignee:California Institute of Technology
Related URLs:
URLURL TypeDescription Patents Patent Office
Perona, Pietro0000-0002-7583-5809
Additional Information:Application filed: May 27, 2016. Patent granted: July 11, 2017.
Record Number:CaltechAUTHORS:20180614-120834409
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:87107
Deposited By: Caroline Murphy
Deposited On:05 Jul 2018 23:12
Last Modified:03 Oct 2019 19:52

Repository Staff Only: item control page