CaltechAUTHORS
  A Caltech Library Service

What you saw is what you will hear: Two new illusions with audiovisual postdictive effects

Stiles, Noelle R. B. and Li, Monica and Levitan, Carmel A. and Kamitani, Yukiyasu and Shimojo, Shinsuke (2018) What you saw is what you will hear: Two new illusions with audiovisual postdictive effects. PLoS ONE, 13 (10). Art. No. 0204217. ISSN 1932-6203. http://resolver.caltech.edu/CaltechAUTHORS:20181008-103232738

[img] PDF - Published Version
Creative Commons Attribution.

3298Kb
[img] Video (MPEG) (S1 Movie. A demonstration of the illusory AV rabbit) - Supplemental Material
Creative Commons Attribution.

5Mb
[img] Video (MPEG) (S2 Movie. A demonstration of the invisible AV rabbit) - Supplemental Material
Creative Commons Attribution.

3662Kb
[img] MS Word (S1 Table. Experiment 1.2 illusory rabbit location) - Supplemental Material
Creative Commons Attribution.

108Kb

Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20181008-103232738

Abstract

Neuroscience investigations are most often focused on the prediction of future perception or decisions based on prior brain states or stimulus presentations. However, the brain can also process information retroactively, such that later stimuli impact conscious percepts of the stimuli that have already occurred (called “postdiction”). Postdictive effects have thus far been mostly unimodal (such as apparent motion), and the models for postdiction have accordingly been limited to early sensory regions of one modality. We have discovered two related multimodal illusions in which audition instigates postdictive changes in visual perception. In the first illusion (called the “Illusory Audiovisual Rabbit”), the location of an illusory flash is influenced by an auditory beep-flash pair that follows the perceived illusory flash. In the second illusion (called the “Invisible Audiovisual Rabbit”), a beep-flash pair following a real flash suppresses the perception of the earlier flash. Thus, we showed experimentally that these two effects are influenced significantly by postdiction. The audiovisual rabbit illusions indicate that postdiction can bridge the senses, uncovering a relatively-neglected yet critical type of neural processing underlying perceptual awareness. Furthermore, these two new illusions broaden the Double Flash Illusion, in which a single real flash is doubled by two sounds. Whereas the double flash indicated that audition can create an illusory flash, these rabbit illusions expand audition’s influence on vision to the suppression of a real flash and the relocation of an illusory flash. These new additions to auditory-visual interactions indicate a spatio-temporally fine-tuned coupling of the senses to generate perception.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1371/journal.pone.0204217DOIArticle
https://doi.org/10.1371/journal.pone.0204217.s001PublisherS1 Movie
https://doi.org/10.1371/journal.pone.0204217.s002PublisherS2 Movie
https://doi.org/10.1371/journal.pone.0204217.s003PublisherS1 Table
https://osf.io/m76ne/?view_only=584e0fc338ee40cd9be0c80e35b3e104Related ItemData
ORCID:
AuthorORCID
Stiles, Noelle R. B.0000-0002-7352-5815
Additional Information:© 2018 Stiles et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Received: July 14, 2017; Accepted: September 5, 2018; Published: October 3, 2018. Data Availability Statement: All data are available from the Open Science Framework (https://osf.io/m76ne/?view_only=584e0fc338ee40cd9be0c80e35b3e104). We are grateful for research grants from the National Institutes of Health, the Philanthropic Educational Organization Scholar Award Program, from the Japan Science and Technology Agency (JST), Core Research for Evolutional Science and Technology (CREST), and from JSPS KAKENHI Grant number JP15H05710. ATR Computational Neuroscience Laboratories did not provided support in the form of salaries and did not have any role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: Professor Kamitani's relationship with ATR Computational Neuroscience Laboratories does not alter our adherence to PLOS ONE policies on sharing data and materials. Author Contributions: Conceptualization: Noelle R. B. Stiles, Monica Li, Carmel A. Levitan, Yukiyasu Kamitani, Shinsuke Shimojo. Data curation: Noelle R. B. Stiles, Monica Li, Carmel A. Levitan. Formal analysis: Monica Li, Carmel A. Levitan. Funding acquisition: Carmel A. Levitan, Shinsuke Shimojo. Investigation: Noelle R. B. Stiles. Methodology: Noelle R. B. Stiles, Monica Li, Carmel A. Levitan, Shinsuke Shimojo. Resources: Shinsuke Shimojo. Software: Monica Li. Supervision: Noelle R. B. Stiles, Yukiyasu Kamitani, Shinsuke Shimojo. Visualization: Noelle R. B. Stiles. Writing ± original draft: Noelle R. B. Stiles. Writing ± review & editing: Noelle R. B. Stiles, Monica Li, Carmel A. Levitan, Yukiyasu Kamitani, Shinsuke Shimojo.
Funders:
Funding AgencyGrant Number
NIHUNSPECIFIED
Philanthropic Educational OrganizationUNSPECIFIED
Japan Science and Technology Agency (JST) Core Research for Evolutional Science and Technology (CREST)UNSPECIFIED
Japan Society for the Promotion of Science (JSPS)JP15H05710
Record Number:CaltechAUTHORS:20181008-103232738
Persistent URL:http://resolver.caltech.edu/CaltechAUTHORS:20181008-103232738
Official Citation:Stiles NRB, Li M, Levitan CA, Kamitani Y, Shimojo S (2018) What you saw is what you will hear: Two new illusions with audiovisual postdictive effects. PLoS ONE 13(10): e0204217. https://doi.org/10.1371/journal.pone.0204217
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:90151
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:08 Oct 2018 17:43
Last Modified:08 Oct 2018 17:43

Repository Staff Only: item control page