Supporting Information
Navalpakkam et al. 10.1073/pnas.0911972107
Decision Models
Here, we explain the derivation of the global likelihood terms in
Eqs.
3
–
5
. Let
P
ð
T
x
¼
H
j
a
!
Þ
denote the posterior probability of
target H
’
s presence at location
x
given the vector of sensory ob-
servations at all locations in the display,
a
!
¼f
a
1
...
a
n
g
. Each
display consists of target H, V, and
n
−
2 distractors. Thus, the
posterior probability of H
’
s presence at location
x
can be expressed
as the joint probability of H
’
s presence at
x
, target V
’
s presence at
any other location
y
≠
x
, and distractor D
’
s presence at the re-
maining locations
z
≠
x
,
y
, marginalized over all choices of
y
. The
posterior probabilities can then be expressed as a product of the
likelihood and prior terms. We assume that target V can occur at
any location
y
≠
x
with equal probability,
P
ð
T
y
¼
V
j
T
x
¼
H
Þ¼
1
n
−
1
.
Thus, we get the following:
P
ð
T
x
¼
H
j
a
!
Þ¼
∑
y
≠
x
P
ð
T
x
¼
H
;
T
y
¼
V
;
T
z
≠
x
;
y
¼
D
j
a
!
Þ
[S1]
P
ð
a
!
j
T
x
¼
H
Þ
P
ð
T
x
¼
H
Þ¼
∑
y
≠
x
P
ð
a
!
j
T
x
¼
H
;
T
y
¼
V
;
T
z
≠
x
;
y
¼
D
Þ
×
P
ð
T
y
¼
V
j
T
x
¼
H
Þ
P
ð
T
x
¼
H
Þ
[S2]
P
ð
a
!
j
T
x
¼
H
Þ¼
1
n
−
1
∑
y
≠
x
P
ð
a
!
j
T
x
¼
H
;
T
y
¼
V
;
T
z
≠
x
;
y
¼
D
Þ
[S3]
In
Eq.
S3
,
P
ð
a
!
j
T
x
¼
H
Þ
denotes the global likelihood of target
H
’
s presence at location
x
, based on the sensory observations at
all locations in the display,
a
!
¼f
a
1
...
a
n
g
. Let
P
(
a
x
|
T
x
=
H
)
denote the local likelihood of H
’
s presence at location
x
based on
the sensory observation at that single location,
a
x
.
The global likelihood term on the right-hand side in
Eq.
S3
can
be expressed as a product of local likelihoods, as shown in
Eq.
S4
. This completes the derivation of Eq.
3
. Eqs.
4
and
5
are
derived similarly.
P
ð
a
!
¼f
a
1
...
a
n
T
x
¼
H
Þ¼
1
n
−
1
∑
y
≠
x
P
ð
a
x
j
T
x
¼
H
Þ
×
P
ð
a
y
j
T
y
¼
V
Þ
Π
z
≠
x
;
y
P
ð
a
z
j
T
z
¼
D
Þ
[S4]
Experiment 1: Orientation
IndividualSubjects
’
DataComparedwithModelPredictions.
Fig. 2
A
–
D
shows data from subject S1. Here, we show the data from the
remaining subjects S2 through S6. As seen in
Figs. S1
–
S5
, be-
havior of all subjects is consistent with the predictions of M3.
Thus, subjects in experiment 1 behave as reward maximizers that
saccade to the location of the maximum expected reward.
In experiment 1 (and others),
fi
xations could land on either
target H, target V, or distractor D.
Fig. S9
shows the distribution
of
fi
xations of subject 1 on target H, target V, and distractor D in
different value and feature-contrast conditions. Note that the
average probability of
fi
xating a distractor is 8.1% (SD = 5.6%)
and that it is not affected by the relative value or feature-contrast
of the targets [no effect of value using paired
t
tests at a sig-
ni
fi
cance level of 0.05; no effect of feature-contrast using paired
t
tests at a signi
fi
cance level of 0.05; no correlations between %
fi
xations on the distractor and target
’
s relative value (
R
2
= 0.01)
or feature-contrast (
R
2
= 0.04)]. For this reason, all subsequent
analyses focus on
fi
xations to one of the targets (e.g., H).
Fig. S9
also shows that in many conditions, subjects chose
fl
exibly be-
tween targets H and V (e.g.,
Fig. S9
,
Upper Left
, when target H is
twice as salient but V is twice as valuable;
Upper Right
, when both
targets are equally salient and valuable;
Lower Left
, when target
H is twice as valuable but V is twice as salient). This shows that
subjects used a dynamic strategy of choosing
fl
exibly different
targets between trials rather than a static strategy that they may
have acquired through ample practice or training in a condition.
Saccade Latency.
How do value and feature-contrast affect latency
to the
fi
rst saccade?
Fig. S8
shows the data pooled over all six
subjects. The data show two main trends: (
i
) saccade latency de-
creases as the target
’
s feature-contrast increases (two-way
ANOVA shows a signi
fi
cant effect of feature-contrast:
F
(6, 18) =
898,
P
<
0.01) until it reaches a mean latency of
≈
330 ms (the
highest latency of 500 ms occurs when the target
’
s feature-contrast
is very low and indicates that it is never
fi
xated within 500 ms), and
(
ii
) saccade latency decreases as the target
’
s relative value in-
creases (two-way ANOVA shows a signi
fi
cant effect of value:
F
(3,
18) = 28,
P
<
0.01). Thus, value and feature-contrast affect the
initial gaze and its latency. The shortest latency of around 330 ms
followed by a
fi
xation duration of at least 100 ms (required to earn
the reward) indicates that subjects mostly made one saccade
during the 500 ms of presentation in experiment 1.
Individual Subjects
’
Data for Experiment 2: Intensity
The reward maximization behavior observed in experiment 1 on
oriented stimuli extends to experiment 2 on luminance stimuli.
Fig. 3 and
Figs. S6
and
S7
show the data from three individual
subjects.
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
1of9
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
% fixat i ons
H
% fixat i ons
H
v
H
/v
V
= 0.5
χ
χ
χ
χ
χ
χ
χ
χχχ
χ
χ
2
(M1)=7.63,
2
(M2)=5483.16,
2
(M3)=1.01
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 1
2
(M1)=6.88,
2
(M2)=46.88,
2
(M3)=2.14
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
% fixations H
% fixations H
v
H
/v
V
= 2
2
(M1)=35.41,
2
(M2)=11.61,
2
(M3)=0.71
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 4
2
(M1)=72.75,
2
(M2)=13.54,
2
(M3)=1.84
A
0
20
40
60
80
100
0
20
40
60
80
100
M3:
2
=0.96, slope=0.96
M2:
2
=0.51, slope=0.57
M1:
2
=0.81, slope=0.94
subject’s detection rates
Predicted detection rates
B
R
R
R
Fig. S1.
Experiment 1, subject S2 (author). (
A
) Subject
’
s data (black dots with error bars denoting SEM) and the psychometric functions predicted by the three
models (M1: light gray, M2: dark gray, M3: red). Each panel denotes a different ratio of values of targets H vs. V. The
χ
2
goodness-of-
fi
t statistic (comparing
how well each model
fi
ts the data) is shown in the title. Across all conditions tested, M3 (the ideal observer)
fi
ts the subject
’
s data better than M1 and M2. (
B
)
Predictions of model M3 correlate well with the subject
’
s data (pooled across all value conditions).
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
2of9
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
% fixations H
% fixations H
% fixations H
% fixations H
v
H
/v
V
= 0.5
χ
χ
χ
χ
χ
χ
χ
χ
χ
χ
χ
χ
2
(M1)=7.08,
2
(M2)=5389.96,
2
(M3)=1.43
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 1
2
(M1)=2.45,
2
(M2)=27.92,
2
(M3)=1.15
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 2
2
(M1)=49.01,
2
(M2)=7.86,
2
(M3)=3.48
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 4
2
(M1)=65.21,
2
(M2)=2.14,
2
(M3)=1.37
A
020406080100
0
20
40
60
80
100
M3:
2
=0.97, slope=1.01
M2:
2
=0.50, slope=0.61
M1:
2
=0.79, slope=0.96
subject’s detection rates
Predicted detection rates
B
R
R
R
Fig. S2.
Experiment 1, subject S3; similar to
Fig. S1
.
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
3of9
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
% fixations H
% fixations H
% fixations H
% fixations H
v
H
/v
V
= 0.5
2
(M1)=9.25,
2
(M2)=3934.10,
2
(M3)=2.17
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 1
2
(M1)=2.62,
2
(M2)=22.32,
2
(M3)=2.97
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 2
2
(M1)=31.13,
2
(M2)=9.93,
2
(M3)=1.43
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 4
2
(M1)=54.49,
2
(M2)=6.00,
2
(M3)=1.76
0
20
40
60
80
100
0
20
40
60
80
100
M3:
2
=0.94, slope=1.00
M2:
2
=0.55, slope=0.64
M1:
2
=0.76, slope=0.95
subject’s detection rates
Predicted detection rates
A
B
χ
χ
χχ
χ
χ
χ
χ
χ
χ
χ
χ
R
R
R
Fig. S3.
Experiment 1, subject S4; similar to
Fig. S1
.
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
4of9
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
feature−contrast H/V
% fixations H
% fixations H
% fixations H
% fixations H
v
H
/v
V
= 0.5
2
(M1)=11.91,
2
(M2)=3341.05,
2
(M3)=2.14
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
feature−contrast H/V
v
H
/v
V
= 1
2
(M1)=2.70,
2
(M2)=21.51,
2
(M3)=1.79
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
v
H
/v
V
= 2
2
(M1)=38.68,
2
(M2)=5.07,
2
(M3)=10.79
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
v
H
/v
V
= 4
2
(M1)=109.08,
2
(M2)=5.84,
2
(M3)=3.58
0
20
40
60
80
100
0
20
40
60
80
100
M3:
2
=0.91, slope=1.02
M2:
2
=0.59, slope=0.70
M1:
2
=0.73, slope=0.97
subject’s detection rates
Predicted detection rates
A
B
χ
χ
χ
χ
χχ
χ
χχ
χ
χ
χ
R
R
R
Fig. S4.
Experiment 1, subject S5; similar to
Fig. S1
.
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
5of9
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
% fixations H
% fixations H
% fixations H
% fixations H
v
H
/v
V
= 0.5
2
(M1)=10.49,
2
(M2)=1835.80,
2
(M3)=2.83
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 1
2
(M1)=2.98,
2
(M2)=30.72,
2
(M3)=4.26
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 2
2
(M1)=28.26,
2
(M2)=9.24,
2
(M3)=0.93
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 4
2
(M1)=74.47,
2
(M2)=7.00,
2
(M3)=2.12
020406080100
0
20
40
60
80
100
M3:
2
=0.92, slope=0.97
M2:
2
=0.55, slope=0.60
M1:
2
=0.74, slope=0.89
subject’s detection rates
Predicted detection rates
A
B
χ
χ
χχχχ
χ
χχ
χ
χ
χ
R
R
R
Fig. S5.
Experiment 1, subject S6; similar to
Fig. S1
.
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
6of9
0.2 0.33 0.50.71 1 1.4 2
5
0
20
40
60
80
100
feature−contrast L/H
% fixations L
% fixations L
% fixations L
v
L
/v
H
= 1
2
(M1)=1.40,
2
(M2)=63.47,
2
(M3)=7.63
0.2 0.33 0.50.71 1 1.4 2
5
0
20
40
60
80
100
feature−contrast L/H
feature−contrast L/H
v
L
/v
H
= 2
2
(M1)=152.72,
2
(M2)=17.66,
2
(M3)=2.44
0.2 0.33 0.50.71 1 1.4 2
5
0
20
40
60
80
100
v
L
/v
H
= 4
2
(M1)=388.99,
2
(M2)=11.60,
2
(M3)=2.42
020406080100
0
20
40
60
80
100
subject’s detection rates
Predicted detection rates
M1:
2
=0.68, slope=0.90
M2:
2
=0
.
68, slope=0.53
M3:
2
=0.95, sl o pe=0.89
A
B
χ
χ
χ
χ
χ
χ
χ
χ
χ
R
R
R
Fig. S6.
Experiment 2, subject S2; similar to
Fig. S1
but using brightness intensity as the feature.
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
7of9
0.2
0.50.71 1 1.4 2
5
0
20
40
60
80
100
feature−contrast L/H
% fixations L
% fixations L
% fixations L
v
L
/v
H
= 1
2
(M1)=6.73,
2
(M2)=45.25,
2
(M3)=1.73
0.2
0.50.71 1 1.4 2
5
0
20
40
60
80
100
feature−contrast L/H
v
L
/v
H
= 2
2
(M1)=186.31,
2
(M2)=13.84,
2
(M3)=0.68
0.2
0.50.71 1 1.4 2
5
0
20
40
60
80
100
feature−contrast L/H
v
L
/v
H
= 4
2
(M1)=325.44,
2
(M2)=10.78,
2
(M3)=5.86
0 20406080100
0
20
40
60
80
100
subject’s detection rates
Predicted detection rates
M1:
2
=0.53, slope=0.87
M2:
2
=0.69, slope=0.57
M3:
2
=0.96, slope=0.94
A
B
χ
χχ
χ
χ
χ
χ
χ
χ
R
R
R
Fig. S7.
Experiment 2, subject S3; similar to
Fig. S1
but using brightness intensity as the feature.
0.059
0.2
0.5
1
2
5
17
320
340
360
380
400
420
440
460
480
500
feature−contrast H/V
RT ( m s)
0.5
1
2
4
v
H
/ v
V
Fig. S8.
Saccadic latency for experiment 1. This
fi
gure shows the time to saccade to the horizontal target as a function of its relative value and feature-
contrast. Saccades become faster as the target
’
s value and feature-contrast increase. RT, reaction time.
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
8of9
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
%fixations
%fixations
%fixations
%fixations
v
H
/v
V
= 0.5
H
V
D
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 1
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 2
0.059
0.2
0.5
1
2
5
17
0
20
40
60
80
100
feature−contrast H/V
v
H
/v
V
= 4
Fig. S9.
Experiment 1, subject S1. Fixations to the distractor (blue), target H (black), and target V (green) for different value and feature-contrast condit
ions in
experiment 1. Although the
fi
xations to the targets vary systematically with value and feature-contrast, the
fi
xations to the distractor appear to be unaffected
(details provided in
SI Text
).
Navalpakkam et al.
www.pnas.org/cgi/content/short/0911972107
9of9