Temporal multiplexing of perception and memory codes in IT cortex
-
1.
California Institute of Technology
Abstract
A central assumption of neuroscience is that long-term memories are represented by the same brain areas that encode sensory stimuli1. Neurons in inferotemporal (IT) cortex represent the sensory percept of visual objects using a distributed axis code2,3,4. Whether and how the same IT neural population represents the long-term memory of visual objects remains unclear. Here we examined how familiar faces are encoded in the IT anterior medial face patch (AM), perirhinal face patch (PR) and temporal pole face patch (TP). In AM and PR we observed that the encoding axis for familiar faces is rotated relative to that for unfamiliar faces at long latency; in TP this memory-related rotation was much weaker. Contrary to previous claims, the relative response magnitude to familiar versus unfamiliar faces was not a stable indicator of familiarity in any patch5,6,7,8,9,10,11. The mechanism underlying the memory-related axis change is likely intrinsic to IT cortex, because inactivation of PR did not affect axis change dynamics in AM. Overall, our results suggest that memories of familiar faces are represented in AM and perirhinal cortex by a distinct long-latency code, explaining how the same cell population can encode both the percept and memory of faces.
Copyright and License
© The Author(s) 2024. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Acknowledgement
This work was supported by NIH (nos. DP1-NS083063 and EY030650-01), the Howard Hughes Medical Institute, the Simons Foundation, the Human Frontiers in Science Program, the Office of Naval Research and the Chen Center for Systems Neuroscience at Caltech. S.F. is supported by the Simons Foundation, the Gatsby Charitable Foundation, the Swartz Foundation and the NSF’s NeuroNex Program (award no. DBI-1707398). We thank K. M. Gothard for sharing monkey face images, D. Chung and V. Tong for assistance with behavioural testing and N. Schweers for assistance with animal training and scanning.
Contributions
L.S. and D.Y.T. conceived the project and designed experiments. L.S. and Y.S. collected data. L.S. and M.K.B. analysed data. L.S., M.K.B. and D.Y.T. interpreted data, with feedback from S.F. L.S. and D.Y.T. wrote the paper, with feedback from S.F., M.K.B. and Y.S.
Data Availability
The dataset of neural responses to screening stimuli and 1,000 monkey faces is available at https://doi.org/10.5281/zenodo.10460607 (ref. 47). Other datasets are available from the PrimFace database (http://visiome.neuroinf.jp/primface), FERET database (https://www.nist.gov/itl/products-and-services/color-feret-database), CVL Face Database (http://www.lrv.fri.uni-lj.si/facedb.html), MR2 face database (https://osf.io/skbq2/), Chicago Face Database (https://www.chicagofaces.org/), CelebA CelebFaces Attributes Dataset (https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html), FEI Face Database (https://fei.edu.br/~cet/facedatabase.html), PICS Psychological Image Collection at Stirling (https://pics.stir.ac.uk), Caltech faces 1999 (https://data.caltech.edu/records/6rjah-hdv18), Essex Face Recognition Data (http://cswww.essex.ac.uk/mv/allfaces/faces95.html), and The MUCT Face Database (www.milbo.org/muct). Source data are provided with this paper.
Extended Data Fig. 2 Visual stimuli.
Extended Data Fig. 4 Main results of experiment 1 computed separately for each animal individually.
Extended Data Fig. 5 Main results of experiment 2 computed separately for each animal individually.
Code Availability
The code that reproduces the core results (Fig. 2b,c and Extended Data Fig. 5a,b) is available at https://doi.org/10.5281/zenodo.10460607 (ref. 47). All other code is available from the corresponding authors on reasonable request.
Conflict of Interest
The authors declare no competing interests.
Files
| Name | Size | Download all |
|---|---|---|
|
md5:889af46378b1f9e92197b2a02a19f5d4
|
16.1 MB | Preview Download |
|
md5:d54f0c89de5ece27c99586159b756374
|
169.3 kB | Preview Download |
|
md5:621a0939cce246f8e77d45e17e936bc4
|
192.1 kB | Download |
|
md5:1f196fd9aed6386125c09affbf3e3018
|
2.7 MB | Download |
|
md5:b8bfba7f40c33b30e63ad946dce63211
|
455.6 kB | Preview Download |
|
md5:cab52b3a9ffdf0a915aff082b28554f5
|
169.2 kB | Preview Download |
|
md5:5a80367e8a2ec99e151e34edaf496f62
|
591.7 kB | Preview Download |
|
md5:81fd69463b0a0bf447fe79f5ef49c79c
|
221.9 kB | Preview Download |
|
md5:3b13d2fdaed56309e3894e2c2e73b916
|
530.8 kB | Preview Download |
|
md5:f33ed25bf8361fa85b723cf889ba6ce7
|
12.7 kB | Download |
|
md5:901a2a81486daecde78a41a80409a768
|
342.7 kB | Preview Download |
|
md5:a5f72bc4ab6f7f4708bba8751ec7a3b0
|
743.1 kB | Preview Download |
|
md5:70bb51a195f9a718aa27f9125b53051a
|
275.6 kB | Preview Download |
|
md5:7654eae414a5a527ad6612dc824c2a11
|
531.0 kB | Download |
|
md5:8bb74d834137a3f8081ac0596da73fde
|
924.0 kB | Download |
|
md5:60fab8ab6e83772abc913bd198acc386
|
1.3 MB | Preview Download |
|
md5:d2d3fde7b86d58db3ade632b530e9747
|
188.1 kB | Preview Download |
Additional details
- ISSN
- 1476-4687
- DP1-NS083063
- National Institutes of Health
- EY030650-01
- National Institutes of Health
- Howard Hughes Medical Institute
- Simons Foundation
- International Human Frontier Science Program Organization
- Office of Naval Research
- Tianqiao and Chrissy Chen Institute for Neuroscience
- California Institute of Technology
- Simons Foundation
- Gatsby Charitable Foundation
- Swartz Foundation
- DBI-1707398
- National Science Foundation
- Caltech groups
- Division of Biology and Biological Engineering, Tianqiao and Chrissy Chen Institute for Neuroscience