NeuroImage
263
(2022)
119585
Contents
lists
available
at
ScienceDirect
NeuroImage
journal
homepage:
www.elsevier.com/locate/neuroimage
Dynamic
neural
reconfiguration
for
distinct
strategies
during
competitive
social
interactions
Ruihan
Yang
a
,
b
,
Yina
Ma
c
,
Bao-Bao
Pan
d
,
Meghana
A.
Bhatt
e
,
Terry
Lohrenz
f
,
Hua-Guang
Gu
d
,
Jonathan
W.
Kanen
g
,
h
,
Colin
F.
Camerer
i
,
j
,
P.
Read
Montague
f
,
k
,
Qiang
Luo
a
,
b
,
l
,
∗
a
National
Clinical
Research
Center
for
Aging
and
Medicine
at
Huashan
Hospital,
Ministry
of
Education-Key
Laboratory
of
Computational
Neuroscience
and
Brain-Inspired
Intelligence,
Institute
of
Science
and
Technology
for
Brain-Inspired
Intelligence,
Fudan
University,
Shanghai
200433,
China
b
State
Key
Laboratory
of
Medical
Neurobiology
and
Ministry
of
Education
Frontiers
Center
for
Brain
Science,
Institutes
of
Brain
Science
and
Human
Phenome
Institute,
Fudan
University,
Shanghai
200032,
China
c
State
Key
Laboratory
of
Cognitive
Neuroscience,
Beijing
Key
Laboratory
of
Brain
Imaging
and
Connectomics,
Learning
and
IDG-McGovern
Institute
for
Brain
Research,
Beijing
Normal
University,
Beijing
100875,
China
d
School
of
Aerospace
Engineering
and
Applied
Mechanics,
Tongji
University,
Shanghai
200092,
China
e
Virginia
Tech
Carilion
Research
Institute,
2
Riverside
Circle,
Roanoke,
VA
24016,
USA
f
Fralin
Biomedical
Research
Institute
at
VTC,
Virginia
Polytechnic
Institute
and
State
University,
Roanoke,
Virginia
g
Department
of
Psychology,
University
of
Cambridge,
Cambridge
CB2
3EB,
UK
h
Behavioural
and
Clinical
Neuroscience
Institute,
University
of
Cambridge,
Cambridge
CB2
3EB,
UK
i
Computation
and
Neural
Systems
Program,
California
Institute
of
Technology,
1200
East
California
Boulevard,
Pasadena,
CA
91125,
USA
j
Department
of
Humanities
and
Social
Sciences,
California
Institute
of
Technology,
1200
East
California
Boulevard,
Pasadena,
CA
91125,
USA
k
Wellcome
Centre
for
Human
Neuroimaging,
University
College
London,
London,
UK
l
Shanghai
Key
Laboratory
of
Mental
Health
and
Psychological
Crisis
Intervention,
School
of
Psychology
and
Cognitive
Science,
East
China
Normal
University,
Shanghai
200241,
China
a
r
t
i
c
l
e
i
n
f
o
Keywords:
Dynamic
behavior
modeling
Dynamic
brain
network
Information
flow
Strategic
deception
a
b
s
t
r
a
c
t
Information
exchange
between
brain
regions
is
key
to
understanding
information
processing
for
social
decision-
making,
but
most
analyses
ignore
its
dynamic
nature.
New
insights
on
this
dynamic
might
help
us
to
uncover
the
neural
correlates
of
social
cognition
in
the
healthy
population
and
also
to
understand
the
malfunctioning
neural
computations
underlying
dysfunctional
social
behavior
in
patients
with
mental
disorders.
In
this
work,
we
used
a
multi-round
bargaining
game
to
detect
switches
between
distinct
bargaining
strategies
in
a
cohort
of
76
healthy
participants.
These
switches
were
uncovered
by
dynamic
behavioral
modeling
using
the
hidden
Markov
model.
Proposing
a
novel
model
of
dynamic
effective
connectivity
to
estimate
the
information
flow
between
key
brain
regions,
we
found
a
stronger
interaction
between
the
right
temporoparietal
junction
(rTPJ)
and
the
right
dorsolateral
prefrontal
cortex
(rDLPFC)
for
the
strategic
deception
compared
with
the
social
heuristic
strategies.
The
level
of
deception
was
associated
with
the
information
flow
from
the
Brodmann
area
10
to
the
rTPJ,
and
this
association
was
modulated
by
the
rTPJ-to-rDLPFC
information
flow.
These
findings
suggest
that
dynamic
bargaining
strategy
is
supported
by
dynamic
reconfiguration
of
the
rDLPFC-and-rTPJ
interaction
during
competitive
social
interactions.
1.
Introduction
Competitive
social
interaction
is
a
common
situation
in
which
people
compete
with
one
another
for
a
finite
resource
or
a
common
objective
Swab
and
Johnson
(2019)
.
When
these
interactions
repeat
many
times,
participants
often
dynamically
switch
between
different
strategies,
such
as
reputation-building
and
reward-collecting,
usually
with
a
long-term
goal
of
maximizing
self-interests
Camerer
and
Weigelt
(1988)
.
These
∗
Corresponding
author
at:
National
Clinical
Research
Center
for
Aging
and
Medicine
at
Huashan
Hospital,
Ministry
of
Education-Key
Laboratory
of
Computational
Neuroscience
and
Brain-Inspired
Intelligence,
Institute
of
Science
and
Technology
for
Brain-Inspired
Intelligence,
Fudan
University,
Shanghai
200433,
China.
E-mail
address:
qluo@fudan.edu.cn
(Q.
Luo)
.
dynamic
behaviors
have
been
widely
observed
and
are
under
exten-
sive
investigation
in
behavioral
economics
research
Camerer
(2003)
and
also
in
mental
health
research
for
patients
with
developmental
and
per-
sonality
disorders
King-Casas
et
al.
(2008)
.
A
question
that
naturally
arose
in
neuroscience
is:
how
are
these
dynamic
strategies
supported
and
constrained
by
underlying
biological
substrates
Montague
(2008)
.
Brain
regions,
such
as
the
right
dorsolateral
prefrontal
cortex
(rDLPFC)
for
calculating
one’s
own
strategy,
and
the
right
temporoparietal
junc-
tion
(rTPJ)
for
taking
the
other’s
perspective,
have
long
been
implicated
in
these
interactions
Glimcher
and
Fehr
(2013)
.
It
has
been
hypothesized
https://doi.org/10.1016/j.neuroimage.2022.119585
.
Received
8
April
2022;
Received
in
revised
form
7
August
2022;
Accepted
22
August
2022
Available
online
25
August
2022.
1053-8119/©2022
The
Authors.
Published
by
Elsevier
Inc.
This
is
an
open
access
article
under
the
CC
BY
license
(
http://creativecommons.org/licenses/by/4.0/
)
R.
Yang,
Y.
Ma,
B.-B.
Pan
et
al.
NeuroImage
263
(2022)
119585
Fig.
1.
Two-party
bargaining
game
and
dynamic
behavior
strategy.
(A)
Task
design:
the
“buyer
”is
given
the
private
value
푣
of
a
hypothetical
object.
He
or
she
is
then
asked
to
“suggest
a
price
푠
”to
the
seller
(values
and
prices
are
integers,
1–10).
The
seller
then
receives
the
suggestion
price
푠
and
is
asked
to
offer
a
price
푝
.
If
the
offered
price
is
less
than
the
private
value
of
the
object,
the
trade
will
be
executed,
and
the
seller
receives
a
reward
of
푝
while
the
buyer
receives
a
reward
of
푣
−
푝
,
otherwise,
the
trade
will
not
occur.
Buyers
and
sellers
do
not
receive
feedback
after
each
trial.
(B)
Dynamic
bargaining
strategy
of
a
buyer
(Subject
ID
64)
during
the
60
rounds.
True
value
푣
(red)
and
suggested
price
푠
(blue)
were
plotted.
Scatter
plots
for
the
true
value
against
the
suggested
price
were
reported
for
each
behavioral
window
together
with
a
least-squares
line
fitted
to
the
data.
(C)
Positive
slope
for
a
bargaining
strategy
of
incrementalist
sharing
the
reward
with
the
seller.
(D)
Negative
slope
for
strategic
deception
trying
to
maximize
the
buyer’s
own
reward.
(E)
Near
zero
slope
for
conservative
who
does
not
communicate
any
information
to
the
seller
during
the
game.
that
these
dynamic
strategies
for
social
interaction
are
supported
by
dy-
namic
reconfiguration
of
the
functional
interactions
among
these
brain
regions
Yang
et
al.
(2020)
.
However,
owing
to
the
technical
limitations
of
commonly
used
dynamic
modeling
approaches
Calhoun
et
al.
(2014)
,
there
have
been
few
studies
on
the
dynamics
of
these
functional
interac-
tions
and
their
behavioral
association
with
strategic
sophistication.
New
insights
on
this
topic
might
help
us
to
uncover
the
neural
correlates
of
social
cognition
in
healthy
population
and
also
to
understand
the
mal-
functioning
neural
computations
underlying
dysfunctional
social
behav-
ior
in
patients
with
mental
disorders
Brüne
and
Brüne-Cohrs
(2006)
.
To
probe
the
neural
bases
of
the
dynamics
in
social
decision-making,
here
we
used
a
self-paced,
multi-round
social
interaction
game
(
Fig.
1
A)
Bhatt
et
al.
(2010)
.
In
each
round
of
the
game,
a
buyer
is
first
informed
by
the
computer
of
the
true
value
of
a
virtual
item
and
then
suggests
a
price
to
the
seller
to
sell
the
item.
The
seller
will
sell
for
any
positive
price.
The
seller
can
infer
the
prior
probability
distribution
of
possible
values
but
is
not
informed
of
the
buyer’s
trial-specific
value.
The
seller
makes
a
price
offer
and
if
the
offer
is
below
the
value,
a
sale
takes
place
(but
this
information
is
hidden
to
the
players,
who
do
not
get
any
feed-
back).
The
final
earnings
from
sales
were
reported
at
the
end
and
paid
to
subjects.
Our
focus
was
on
the
strategies
that
buyers
used
to
suggest
prices
based
on
their
private
trial-specific
values.
In
our
sample,
we
observed
three
types
of
bargaining
strategies
Bhatt
et
al.
(2010)
:
1)
“conserva-
tives
”whose
suggested
prices
revealed
no
information
to
their
partners;
2)
“incrementalists
”who
anchored
their
social
signals
(i.e.
the
suggested
prices)
to
the
true
values
of
the
items
(as
evidenced
by
a
high
correlation
between
values
and
prices);
and
3)
“strategists
”who
used
a
more
sophis-
ticated
strategy
by
mimicking
the
incrementalists.
That
is,
the
strategists
generated
a
series
of
prices
with
variability
similar
to
the
prices
sug-
gested
by
incrementalists,
in
order
to
build
a
reputation
in
their
part-
ners’
minds
that
their
prices
were
revealing
information
about
value.
However,
the
strategists
suggested
low
prices
for
the
most
highly
val-
ued
items
(to
earn
a
lot
in
those
trials,
i.e.
reward-collecting)
and
high
prices
for
low-value
items
(which
are
not
very
profitable
but
necessary
for
reputation-building).
Their
values
and
prices
are
therefore
negatively
correlated.
Theoretically,
the
existence
of
these
three
strategy
types
has
also
been
predicted
by
a
Bayesian
model
of
belief
formation
with
each
type
possessing
different
depths
of
theory
of
mind,
and
no
other
strate-
gies
has
been
predicted
Bhatt
et
al.
(2010)
.
In
a
previous
analysis
of
this
game,
we
considered
the
last
30
rounds
of
the
game
to
be
revealing
a
single
stable
strategy
Bhatt
et
al.
(2010)
.
Each
subject
was
classified
into
one
of
the
above
three
strategic
groups
by
the
extent
to
which
their
suggested
prices
revealed
the
true
value
of
the
bargaining
item
Bhatt
et
al.
(2010)
.
Compared
with
the
other
two
groups,
we
found
greater
activity
of
the
left
rostral
prefrontal
2
R.
Yang,
Y.
Ma,
B.-B.
Pan
et
al.
NeuroImage
263
(2022)
119585
cortex
[rPFC
or
Brodmann
area
10
(BA10)]
in
the
strategic
group
Bhatt
et
al.
(2010)
,
suggesting
that
long-term
goal
maintenance
was
necessary
for
the
strategic
deception.
The
stronger
information
flows
from
both
dorsal
anterior
cingulate
cortex
and
retrosplenial
cortex
to
the
BA10
were
associated
with
a
higher
level
of
deception
during
the
game
Luo
et
al.
(2017)
,
which
further
highlighted
the
involvement
of
the
cognitive
control
systems
for
the
strategic
deception.
Apart
from
BA10,
the
rDLPFC
has
been
associated
with
self-interested
behavior,
as
its
cor-
tical
thickness
has
been
shown
to
be
negatively
associated
with
prosocial
giving
to
strangers
in
the
dictator
game,
but
not
in
the
ultimatum
game
Yamagishi
et
al.
(2016)
.
There
is
also
evidence
that
rTPJ
is
essential
for
integrating
others’
beliefs
into
one’s
own
strategic
choice,
since
its
causal
interruption
by
repetitive
transcranial
magnetic
stimulation
(rTMS)
re-
duced
the
ability
to
model
the
other’s
belief
Hill
et
al.
(2017)
.
However,
the
functional
role
of
the
rDLPFC-and-rTPJ
interaction
remains
unclear,
mainly
owing
to
its
dynamic
nature
during
competitive
social
interac-
tions.
In
our
sample,
the
rTPJ
was
dynamically
engaged
in
the
strategic
deception
Bhatt
et
al.
(2010)
,
i.e.
the
rTPJ
became
more
activated
when
the
strategists
switched
from
reputation-building
to
reward-collecting.
Not
only
was
the
engagement
of
rTPJ
dynamic,
but
also
the
bargain-
ing
strategy
of
the
sellers
was
dynamically
switched
during
the
game
(
Fig.
1
B–E).
This
dynamic
switch
of
strategy
might
be
supported
by
the
corresponding
dynamic
reconfiguration
of
the
rDLPFC-and-rTPJ
interac-
tion.
Therefore,
such
a
highly
time-varying
interaction
requires
a
more
dedicated
model
to
reveal
its
dynamics.
Unlike
previous
studies,
which
assumed
that
one
participant
could
have
only
one
strategy
during
the
whole
game,
we
relaxed
this
assump-
tion
by
investigating
the
dynamic
switches
between
strategies
from
trial
to
trial
using
a
hidden
Markov
model
(HMM).
The
hidden
state
at
each
trial
was
defined
as
one
of
the
three
strategies
introduced
above,
includ-
ing
the
conservative,
incremental,
and
strategic
strategies.
The
Markov
property
was
assumed:
given
the
state
of
the
current
trial,
the
state
of
the
next
trial
would
be
independent
of
the
previous
trials.
The
observation
at
each
trial
was
calculated
from
the
association
between
the
suggested
prices
and
the
true
values
in
seven
adjacent
trials
that
are
centering
at
the
current
one.
When
the
adjacent
trials
shared
the
same
hidden
state,
they
naturally
constituted
a
behavioral
window
of
time
when
the
sellers
adopted
the
same
bargaining
strategy.
Next,
we
proposed
a
novel
approach,
namely
time-varying
Granger
causality
with
signal-dependent
noise
(tvGCSDN),
to
estimate
the
dy-
namic
effective
connectivity
between
the
key
brain
regions
at
each
round
(Supplementary
Method
S1).
The
proposed
algorithm
has
the
fol-
lowing
two
main
advantages:
1)
Instead
of
setting
a
window
length
for
the
sliding-window
algorithms
in
most
of
the
dynamic
functional
con-
nectivity
analyses
Preti
et
al.
(2017)
;
Simony
et
al.
(2016)
,
tvGCSDN
makes
an
estimation
at
each
trial
by
borrowing
the
strength
of
functional
magnetic
resonance
imaging
(fMRI)
data
during
the
whole
bargain-
ing
game.
2)
tvGCSDN
is
applicable
to
systems
with
signal-dependent
noise
which
violated
the
assumption
of
Gaussian
white
noise
as-
sumed
by
most
of
the
previous
dynamic
effective
connectivity
mod-
els
Havlicek
et
al.
(2010)
;
Ryali
et
al.
(2011)
;
Sato
et
al.
(2006)
.
Signal-dependent
noise
is
common
in
neural
systems
and
has
been
detected
in
both
physiological
recordings
Harris
and
Wolpert
(1998)
;
Luo
et
al.
(2011)
;
Phan
et
al.
(2019)
and
fMRI
time-series
signals
Anika
et
al.
(2020)
;
Luo
et
al.
(2013,
2017,
2020)
.
We
proved
math-
ematically
that
the
mis-specification
of
the
time-invariant
model
to
a
dynamic
system
underestimates
the
effective
connectivity
(Supplemen-
tary
Method
S1).
We
also
demonstrated
by
simulations
that
the
pro-
posed
tvGCSDN
could
track
the
time-varying
parameters
of
the
dynamic
systems
both
with
and
without
signal-dependent
noise
(Supplementary
Method
S2,
Figs.
S1
and
S2).
The
strength
of
the
effective
connectiv-
ity
estimated
by
Granger
causality
has
been
proven
to
be
equivalent
to
a
measurement
of
the
information
flow
from
the
cause
to
the
effect
Barnett
et
al.
(2009)
.
This
equivalence
enabled
us
to
test
the
behav-
ioral
associations
of
the
estimated
information
flows
between
the
key
brain
regions
in
relation
to
strategic
sophistication
during
the
bargain-
ing
game.
2.
Results
2.1.
Dynamic
switches
between
strategies
uncovered
by
HMM
According
to
the
hidden
states
(i.e.
the
bargaining
strategies)
de-
coded
by
the
HMM
(
Fig.
2
A;
Supplementary
Method
S3),
we
found
three
types
of
behavioral
windows,
including
80
incremental
windows,
28
conservative
windows,
and
28
strategic
windows.
In
total,
31.6%
of
par-
ticipants
(n
=
24)
switched
their
strategies
during
the
game
with
a
mean
[SD]
number
of
switches
1.5
[0.67]
times
(
Fig.
2
B).
We
detected
35
tran-
sitions
between
strategies,
including
8
incremental
to
strategic,
12
incre-
mental
to
conservative,
4
conservative
to
strategic,
10
conservative
to
incremental,
and
1
strategic
to
incremental
transitions.
Compared
with
our
previous
time-invariant
behavioral
groupings
using
only
the
last
30
trials
(
Bhatt
et
al.,
2010
;
Fig.
S3A-B),
we
re-classified
13.4%
of
trials
(
푛
=
305
)
into
a
different
behavioral
category
and
discarded
15.7%
of
trials
(
푛
=
357
)
as
unstable.
To
characterize
each
behavioral
window,
we
fitted
a
linear
regres-
sion
model
using
the
true
value
to
predict
the
suggested
price.
The
slope
of
this
regression
model,
which
reflected
the
way
in
which
buyers
re-
vealed
the
information
about
values
of
the
items
to
sellers
during
the
game,
was
used
as
a
behavioral
parameter
for
the
pattern
of
informa-
tion
revelation
(
IR
).
As
expected,
we
found
that
the
conservative
win-
dows
had
the
IRs
close
to
zeros
(Mean
±
SD
=
0
.
13
±
0
.
11
)
with
low
model
fits
(
푅
2
=
0
.
13
±
0
.
11
),
the
incremental
windows
showed
posi-
tive
IRs
(
0
.
49
±
0
.
19
)
with
good
fits
(
0
.
73
±
0
.
15
),
and
the
strategic
win-
dows
exhibited
negative
IRs
(
−0
.
59
±
0
.
26
)
with
good
fits
(
0
.
46
±
0
.
20
;
Fig.
2
C).
The
values
of
the
virtual
items,
the
starting
time,
and
the
length
of
the
behavioral
window
were
compared
among
three
types
of
behavioral
windows,
and
no
significant
difference
was
found
(Table
S1).
We
also
found
that
buyers
with
older
ages
had
more
incremental
windows
compared
with
those
with
younger
ages
(
푟
=
0
.
36
,
푝
=
0
.
0015
),
while
females
had
more
incremental
windows
compared
with
males
(
푟
=
−0
.
39
,
푝
=
0
.
0004
;
Table
S2).
In
the
estimated
initial
distribution
of
HMM,
we
found
that
the
prob-
ability
of
using
the
incremental
strategy
was
58%,
using
the
conservative
strategy
was
37%,
and
using
the
strategic
strategy
was
5%
(
Fig.
2
D).
As
estimated
by
the
transition
matrix,
the
probabilities
of
the
incremental-
to-conservative
and
the
conservative-to-strategic
transitions
were
0.03
and
0.05,
respectively
(
Fig.
2
A).
As
predicted
by
59
repeated
transitions
from
the
initial
distribution,
the
probability
of
adopting
the
strategic
strategy
significantly
increased
to
26%,
while
the
probabilities
of
the
incremental
and
the
conservative
strategies
decreased
by
7
and
14
per-
centage
points,
respectively
(
Fig.
2
D).
Similar
trends
were
observed
by
the
end
of
the
game,
as
30%
of
participants
used
the
strategic
decep-
tion
while
the
percentages
of
participants
using
the
conservative
and
the
incremental
strategies
decreased
by
14
and
7
percentage
points,
re-
spectively.
Evaluating
the
quality
of
clustering
by
the
Davies–Bouldin
Index
(DBI;
the
smaller
the
DBI
the
better
the
clustering
quality
Davies
and
Bouldin,
1979
),
we
found
that
the
time-varying
grouping
in
the
cur-
rent
study
had
a
better
clustering
quality
(
DBI
1
=
0
.
58
)
than
a
time-
invariant
grouping
reported
previously
(
Bhatt
et
al.,
2010
;
DBI
2
=
0
.
64
;
the
95%
confidence
interval
of
DBI
2
−
DBI
1
was
[0
.
055
,
0
.
062]
estab-
lished
by
3000
bootstraps).
This
advantage
remained
to
be
significant
when
evaluated
by
the
Calinski-Harabasz
Index
(CH;
the
bigger
the
CH
the
better
the
clustering
quality
Cali
ń
ski
and
Harabasz,
1974
).
The
CH
of
the
current
clustering
(
CH
1
)
is
264.9
and
the
previous
clustering
(
CH
2
)
is
173.2.
The
95%
confidence
interval
of
CH
1
−
CH
2
was
[20
.
6
,
171
.
3]
,
es-
tablished
by
3000
bootstraps.
This
might
be
important
for
the
dynamic
effective
connectivity
analysis,
since
a
better
definition
of
the
behavioral
window
could
better
identify
dynamic
activity
between
brain
regions.
3