Enhanced control of a brain–computer interface by tetraplegic participants via neural-network-mediated feature extraction
-
1.
California Institute of Technology
-
2.
University of Southern California
-
3.
Blackrock Microsystems (United States)
- 4. UT Southern Medical Center, Dallas, TX, USA
-
5.
University of California, Los Angeles
Abstract
To infer intent, brain–computer interfaces must extract features that accurately estimate neural activity. However, the degradation of signal quality over time hinders the use of feature-engineering techniques to recover functional information. By using neural data recorded from electrode arrays implanted in the cortices of three human participants, here we show that a convolutional neural network can be used to map electrical signals to neural features by jointly optimizing feature extraction and decoding under the constraint that all the electrodes must use the same neural-network parameters. In all three participants, the neural network led to offline and online performance improvements in a cursor-control task across all metrics, outperforming the rate of threshold crossings and wavelet decomposition of the broadband neural data (among other feature-extraction techniques). We also show that the trained neural network can be used without modification for new datasets, brain areas and participants.
Copyright and License (English)
© 2024, The Author(s), under exclusive licence to Springer Nature Limited
Acknowledgement (English)
We thank K. Pejsa for her help in running recording sessions and JJ, EGS and NS for accepting to participate in this research. We also thank D. Tang for his contributions to online implementation and stability testing of FENet.
Funding (English)
Funding for this Institutional Review Board- and FDA-approved work has been provided by the National Institute of Health grant R01EY015545 (R.A.A., T.A., N.P.), Tianqiao and Chrissy Chen Brain–machine Interface Center at Caltech (R.A.A., T.A.) and Boswell Foundation (R.A.A.). B.H. and A.E. were supported by the Center for Sensing to Intelligence (S2I), Braun Foundation, Chen Institute for Neuroscience and Heritage Medical Research Institute.
Contributions (English)
B.H. and T.A. developed FENet, analysed the results and wrote the paper. T.A. conceptualized the study and provided mentorship. B.H. implemented FENet. B.H., T.A., S.K. and C.G. contributed to experimental design and analysis. J.A.G.d.L. contributed to data recording. A.Y.H. contributed to hyperparameter optimization of FENet. N.P. conducted surgery on human participants and was responsible for participants’ care. R.A.A. and A.E. provided mentorship and supervised the research. All authors reviewed and modified the paper.
Data Availability (English)
The behavioural and neurophysiological data are archived in the Division of Biology and Biological Engineering at the California Institute of Technology. The broadband neural data are confidential and hence cannot be publicly shared. The raw and analysed data generated during the study are available for research purposes from the corresponding authors on reasonable request.
Code Availability (English)
The codes used for training and inference of FENet are available via GitHub at https://github.com/BenyaminHaghi/FENet. Codes used for analysing and displaying the results presented in this study are available from the corresponding authors on reasonable request.
Supplemental Material
- Supplementary Information: Supplementary figures and video captions.
- Supplementary Video 1: Closed-loop test for centre-out task.
- Supplementary Video 2:Closed-loop test for the grid task.
- Supplementary Video 3:Closed-loop test for centre-out task.
- Supplementary Video 4:Closed-loop test for the grid task.
Files
| Name | Size | Download all |
|---|---|---|
|
md5:54550c74b4c61fa9de55324bf1f83d03
|
646.9 MB | Download |
|
md5:5ca0ce78048a5ab50b2e2e117665970a
|
433.0 MB | Download |
|
md5:d48f29d9250bd559eff389b1721cfa36
|
800.3 MB | Download |
|
md5:b479bf7a3207208094e539d439dbbe0b
|
7.6 MB | Preview Download |
|
md5:84fda69faa5d4f03a2cda6d5c0d00348
|
1.2 MB | Preview Download |
Additional details
- National Institutes of Health
- R01EY015545
- Accepted
-
2024-10-27Accepted
- Available
-
2024-12-06Published online
- Caltech groups
- Division of Biology and Biological Engineering
- Publication Status
- Published