A Caltech Library Service

Using semantic content as cues for better scanpath prediction

Cerf, Moran and Frady, E. Paxon and Koch, Christof (2008) Using semantic content as cues for better scanpath prediction. In: ETRA '08 Proceedings of the 2008 symposium on Eye tracking research & applications. ACM , New York, NY, pp. 143-146. ISBN 978-1-59593-982-1.

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item:


Under natural viewing conditions, human observers use shifts in gaze to allocate processing resources to subsets of the visual input. There are many computational models that try to predict these shifts in eye movement and attention. Although the important role of high level stimulus properties (e.g., semantic information) stands undisputed, most models are based solely on low-level image properties. We here demonstrate that a combined model of high-level object detection and low-level saliency significantly outperforms a low-level saliency model in predicting locations humans fixate on. The data is based on eye-movement recordings of humans observing photographs of natural scenes, which contained one of the following high-level stimuli: faces, text, scrambled text or cell phones. We show that observers - even when not instructed to look for anything particular, fixate on a face with a probability of over 80% within their first two fixations, on text and scrambled text with a probability of over 65.1% and 57.9% respectively, and on cell phones with probability of 8.3%. This suggests that content with meaningful semantic information is significantly more likely to be seen earlier. Adding regions of interest (ROI), which depict the locations of the high-level meaningful features, significantly improves the prediction of a saliency model for stimuli with high semantic importance, while it has little effect for an object with no semantic meaning.

Item Type:Book Section
Related URLs:
URLURL TypeDescription
Koch, Christof0000-0001-6482-8067
Additional Information:© 2008 ACM.
Group:Koch Laboratory (KLAB)
Subject Keywords:Eye Tracking, Psychophysics, Natural Scenes
Classification Code:I.4.8 [Image Processing and Computer Vi- sion]: Scene Analysis—Tracking; I.6.4 [Simulation and Modeling]: Model Validation and Analysis
Record Number:CaltechAUTHORS:20160815-155309063
Persistent URL:
Official Citation:Moran Cerf, E. Paxon Frady, and Christof Koch. 2008. Using semantic content as cues for better scanpath prediction. In Proceedings of the 2008 symposium on Eye tracking research & applications (ETRA '08). ACM, New York, NY, USA, 143-146. DOI=
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:69634
Deposited On:16 Aug 2016 18:35
Last Modified:11 Nov 2021 04:17

Repository Staff Only: item control page