A Caltech Library Service

Teaching categories to human learners with visual explanations

Mac Aodha, Oisin and Su, Shihan and Chen, Yuxin and Perona, Pietro and Yue, Yisong (2018) Teaching categories to human learners with visual explanations. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE , Piscataway, NJ, pp. 3820-3828. ISBN 978-1-5386-6420-9.

[img] PDF - Accepted Version
See Usage Policy.


Use this Persistent URL to link to this item:


We study the problem of computer-assisted teaching with explanations. Conventional approaches for machine teaching typically only provide feedback at the instance level e.g., the category or label of the instance. However, it is intuitive that clear explanations from a knowledgeable teacher can significantly improve a student's ability to learn a new concept. To address these existing limitations, we propose a teaching framework that provides interpretable explanations as feedback and models how the learner incorporates this additional information. In the case of images, we show that we can automatically generate explanations that highlight the parts of the image that are responsible for the class label. Experiments on human learners illustrate that, on average, participants achieve better test set performance on challenging categorization tasks when taught with our interpretable approach compared to existing methods.

Item Type:Book Section
Related URLs:
URLURL TypeDescription Paper
Mac Aodha, Oisin0000-0002-5787-5073
Perona, Pietro0000-0002-7583-5809
Yue, Yisong0000-0001-9127-1989
Additional Information:© 2018 IEEE. We would like to thank Google for their gift to the Visipedia project, AWS Research Credits, Bloomberg, Northrop Grumman, and the Swiss NSF for their Early Mobility Postdoctoral Fellowship. Thanks also to Kareem Moussa for providing the OCT dataset and to Kun ho Kim for helping generate crowd embeddings.
Funding AgencyGrant Number
Amazon Web ServicesUNSPECIFIED
Bloomberg Data ScienceUNSPECIFIED
Northrop GrummanUNSPECIFIED
Swiss National Science Foundation (SNSF)UNSPECIFIED
Record Number:CaltechAUTHORS:20180613-143607692
Persistent URL:
Official Citation:O. M. Aodha, S. Su, Y. Chen, P. Perona and Y. Yue, "Teaching Categories to Human Learners with Visual Explanations," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 3820-3828. doi: 10.1109/CVPR.2018.00402
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:87078
Deposited By: Caroline Murphy
Deposited On:13 Jun 2018 21:47
Last Modified:09 Mar 2020 13:19

Repository Staff Only: item control page