Causal inference in multisensory
perception
Supporting Information
Comparison with other models
In recent models of cue combination the response distribution
ˆˆ
(, | , )
VAVA
pssss
is treated
as the posterior distribution
(, | , )
VAVA
pssxx
. This treatment leads to a significant error
for non-Gaussian priors. In this paper we corr
ectly deal with this issue by marginalizing
over the latent variables
A
x
and
V
x
:
()()
(
)
(
)
ˆˆ
ˆˆ
,|,
,|,
|
|
VAVAVAVAVVAAVA
psssspssxxpxspxsdxdx
=
∫∫
.
These response distributions
were obtained through simulation.
The results reported for these other models
here correspond to the
corrected versions of
these models.
1
Priors of other models
Recently, models with so
-called interaction priors have been proposed
, which define
explicit interactions between the cues. These models
do not assume full integration and
thus lead to much better predictions than
the traditional forced-fusion model. While the
model by Shams et al. was the first of such
models, we will not make a comparison with
that model here as it has ma
ny more free parameters than
the causal inference model (and
in fact it may be seen as a s
uperset of all the models tested
here). Table 1 in the main text
analyses how well each of th
e models predicts the data.
Given an interaction prior
()
,
AV
pss
, we obtain the bisensor
y posterior distribution
through Bayes’ rule,
()()()
(
)
,|,
,
|
|
AVAVAVAAVV
pssxxpsspxspxs
∝
.
In earlier models (Bresciani et al.; Roach et
al.), the interaction prio
r is assumed to take a
specific form. In Bresciani et al., it is
a Gaussian ridge on the diagonal with width
coupling
σ
:
()
()
2
2
coupling
2
,
AV
ss
AV
psse
σ
−
−
∝
A straightforward computation then allows
us to obtain the auditory posterior:
(
)
(
)
(
)
(
)
()
()()
()
()
()
()
(
)
()
()
2
2
coupling
2
coupling
22
coupling
1
22 2
coupling
1
22
2
2
coupling
coupli
|,
,
|
|
||
|;,
;,
;,
;,
1
;,
AV
AAVAVAAVVV
ss
AAVVV
AAVAVVVV
AAAAVV
AAVV
A
AV
A
psxxpsspxspxsds
epxspxsds
pxsNssNsxds
NsxNsx
xx
Ns
σ
σσ
σσσ
σσσ
σσ σ
σσ
−
−
−
−
−
−
−
∝
∝
∝
=+
++
∝
++
+
∫
∫
∫
()
1
22
ng
V
σ
−
⎛⎞
⎜⎟
⎜⎟
⎜⎟
+
⎝⎠
where
()
;,
Nx
μ
σ
is the value of the norma
l distribution with mean
μ
and standard
deviation
σ
evaluated at
x
. Because the posterior distribut
ion is a pure Gaussian, it can
be identified with the respon
se distribution. The expression
for the visual distribution is
obtained by interchanging
V
and
A
.
2
In the work by Roach et al. [1], a more
general form of interaction prior is used,
consisting of the same Gaus
sian ridge on the
diagonal, but added
to that a constant
background
ω
:
()
()
2
2
coupling
2
,
AV
ss
AV
psse
σ
ω
−
−
∝+
This gives rise to the following posterior distribution:
()
()
()()
()() ()
()
()
()
()
(
)
()
2
2
coupling
2
2
coupling
coupling
222
coupling
coupling
couplin
|,
|
|
||
|2
;,
;,
;,
2
;,
;,
;,
2
AV
ss
AAVAAVVV
AAVVVAAVAVVVV
AAAAAAAVV
AAA
psxxepxspxsds
pxspxsdspxsNssNsxds
NsxNsxNsx
Nsx
σ
ω
ωπσσσ
ωσπσ
σ σσ
ωσ
πσ
−
−
⎛⎞
⎜⎟
∝+
⎜⎟
⎜⎟
⎝⎠
=+
=+
+
=+
+
∫
∫∫
(
)
()
()
()
1
22 2
coupling
2222
g
coupling
1
1
22
2
22
2
coupling
coupling
1
;,
;
,
AAVV
AVVAA
AV
AV
xx
NxxNs
σσσ
σσσ
σσ σ
σσ σ
−
−
−
−
−
−
⎛⎞
++
⎜⎟
++
⎜⎟
++
⎜⎟
++
⎝⎠
The best estimate under is the mean of this distribution, which is
(
)
(
)
()
(
)
1
22 2
coupling
2222
coupling
coupling
1
22
2
coupling
2222
coupling
coupling
2;,
ˆ
2;,
AAVV
AAVVA
AV
A
AVVA
xx
xNxx
s
Nxx
σσσ
ωπσ
σσσ
σσ σ
ωπσ
σ σσ
−
−
−
−
++
+++
++
=
+++
The expression for the visual estim
ate is obtained by interchanging
V
and
A
. This model
is more similar to the causal inference
model, because the estimates are nonlinear
functions of
V
x
and
A
x
. Because of that reas
on as well, the res
ponse distribution can now
no longer be identified w
ith the response distribution.
There is a direct link between the caus
al inference model and these models with
interaction priors. The causal inference
model can be recast
as a model with an
interaction prior by integrating out the latent variable
C
:
() ( )()
(
)
(
)
(
)
mmon
common
,1
VAcoVAAVA
psspsspsppsps
δ
=−+−
.
All predictions about position
estimates with this model are retained. However, this
model no longer explicitly represents whether
there is a single cause or alternatively two
independent causes. This explains why t
hose models are relatively successful at
explaining the data.
3
Fig 1S shows the priors obt
ained from the models disc
ussed here, after fitting
their parameters to optimally describe the data.
Bias as a function of disparity
In the main text we have used the probability
of the data given the model as a measure for
the quality of each model. We
found that the causal infere
nce model best explains the
data. Although such inferential
statistics are a good tool to compare models, we are also
interested in the question why the causal
inference model performs better. To get an
understanding of the differences we turn to the graph showing bias as a function of
disparity
(Fig. 2S)
.
The data shows that the bias decreases with
increasing spatial
disparity. The further the distance between vi
sual and auditory stimuli, the smaller is the
influence of vision on audition. This result
is naturally predicted by the causal inference
model; larger discrepancies make
the single cause model less likely as it needs to assume
large noise values that are unlikely. The joint
prior used by Bresciani et al. predicts a
bias that is largely invarian
t to the disparity. However,
lacking a way to represent that
two cues may be entirely independent, it unde
restimates the derivative of the bias graph.
References
4
Figure Legends:
Figure 1, supporting information:
The interaction priors when fit to our dataset are
shown for the causal inference model, the Roac
h et al. and the Bres
ciani et al. priors.
Figure 2, supporting information:
The average auditory bias
ˆ
AA
VA
ss
ss
−
−
, i.e. the relative
influence of the visual position
on the perceived auditory pos
ition, is shown as a function
of the absolute spatial disparit
y (solid line, as in Fig. 2 ma
in text) along with the model
predictions (dashed lines).
Red: causal inference model.
Green: behavior
derived from
using the Roach et al prior. Purple: behaviour
derived from using the Bresciani et al prior.
5