CaltechAUTHORS
  A Caltech Library Service

A map of object space in primate inferotemporal cortex

Bao, Pinglei and She, Liang and McGill, Mason and Tsao, Doris Y. (2020) A map of object space in primate inferotemporal cortex. Nature, 583 (7814). pp. 103-108. ISSN 0028-0836. https://resolver.caltech.edu/CaltechAUTHORS:20200224-161613363

[img] Image (JPEG) (Extended Data Fig. 1: Time courses from NML1-3 during microstimulation of NML2) - Supplemental Material
See Usage Policy.

138Kb
[img] Image (JPEG) (Extended Data Fig. 2 Stimuli used in electrophysiological recordings) - Supplemental Material
See Usage Policy.

195Kb
[img] Image (JPEG) (Extended Data Fig. 3: Additional neuronal response properties from different patches) - Supplemental Material
See Usage Policy.

475Kb
[img] Image (JPEG) (Extended Data Fig. 4: Building an object space using a deep network) - Supplemental Material
See Usage Policy.

386Kb
[img] Image (JPEG) (Extended Data Fig. 5: Neurons across IT perform axis coding) - Supplemental Material
See Usage Policy.

283Kb
[img] Image (JPEG) (Extended Data Fig. 6: Similar functional organization is observed using a different stimulus set) - Supplemental Material
See Usage Policy.

395Kb
[img] Image (JPEG) (Extended Data Fig. 7: Response time courses from the four IT networks spanning object space) - Supplemental Material
See Usage Policy.

196Kb
[img] Image (JPEG) (Extended Data Fig. 8: Searching for substructure within patches) - Supplemental Material
See Usage Policy.

387Kb
[img] Image (JPEG) (Extended Data Fig. 9: The object space model parsimoniously explains previous accounts of IT organization) - Supplemental Material
See Usage Policy.

322Kb
[img] Image (JPEG) (Extended Data Fig. 10: Object space dimensions are a better descriptor of response selectivity in the body patch than category labels) - Supplemental Material
See Usage Policy.

338Kb
[img] Image (JPEG) (Extended Data Fig. 11: Object decoding and recovery of images by searching a large auxiliary object database) - Supplemental Material
See Usage Policy.

325Kb
[img] PDF (Supplementary Notes and Supplementary Tables 1-2) - Supplemental Material
See Usage Policy.

730Kb
[img] PDF (Reporting Summary) - Supplemental Material
See Usage Policy.

73Kb

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200224-161613363

Abstract

The inferotemporal (IT) cortex is responsible for object recognition, but it is unclear how the representation of visual objects is organized in this part of the brain. Areas that are selective for categories such as faces, bodies, and scenes have been found, but large parts of IT cortex lack any known specialization, raising the question of what general principle governs IT organization. Here we used functional MRI, microstimulation, electrophysiology, and deep networks to investigate the organization of macaque IT cortex. We built a low-dimensional object space to describe general objects using a feedforward deep neural network trained on object classification. Responses of IT cells to a large set of objects revealed that single IT cells project incoming objects onto specific axes of this space. Anatomically, cells were clustered into four networks according to the first two components of their preferred axes, forming a map of object space. This map was repeated across three hierarchical stages of increasing view invariance, and cells that comprised these maps collectively harboured sufficient coding capacity to approximately reconstruct objects. These results provide a unified picture of IT organization in which category-selective regions are part of a coarse map of object space whose dimensions can be extracted from a deep network.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1038/s41586-020-2350-5DOIArticle
https://rdcu.be/b4BkoPublisherFree ReadCube access
ORCID:
AuthorORCID
Bao, Pinglei0000-0001-6643-2904
Tsao, Doris Y.0000-0003-1083-1919
Additional Information:© 2020 Springer Nature Limited. Received 21 January 2019; Accepted 17 March 2020; Published 03 June 2020. This work was supported by NIH (DP1-NS083063, R01-EY030650), the Howard Hughes Medical Institute, and the Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech. We thank A. Flores for technical support, and members of the Tsao laboratory, N. Kanwisher, A. Kennedy, S. Kornblith, and A. Tsao for critical comments. Author Contributions: P.B. and D.Y.T. designed the experiments, P.B. and L.S. collected the data, and P.B. analysed the data. M.M. provided technical advice on neural networks. P.B. and D.Y.T. interpreted the data and wrote the paper. Data availability: The data that support the findings of this study are available from the lead corresponding author (D.Y.T.) upon reasonable request. The authors declare no competing interests.
Group:Tianqiao and Chrissy Chen Institute for Neuroscience
Funders:
Funding AgencyGrant Number
NIHDP1-NS083063
NIHR01-EY030650
Howard Hughes Medical Institute (HHMI)UNSPECIFIED
Tianqiao and Chrissy Chen Institute for NeuroscienceUNSPECIFIED
Issue or Number:7814
Record Number:CaltechAUTHORS:20200224-161613363
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200224-161613363
Official Citation:Bao, P., She, L., McGill, M. et al. A map of object space in primate inferotemporal cortex. Nature 583, 103–108 (2020). https://doi.org/10.1038/s41586-020-2350-5
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:101514
Collection:CaltechAUTHORS
Deposited By: George Porter
Deposited On:27 Apr 2020 18:31
Last Modified:01 Jul 2020 16:31

Repository Staff Only: item control page