Skip to main content
2025 Neural Interfaces Meeting Main banner
Unparalleled proactive care. Optimizing the SCS experience. Website: neuro.biotronik.com

(34) Large scale deep network models for intracortical motor decoding and control

Thursday, June 12, 2025
5:10 PM - 5:50 PM ET
Location: Sky View
Introduction: Neural data collection is growing rapidly, both in the complexity of individual datasets and the total number of datasets recorded across people and devices. In many fields, similarly abundant data are being used to train large scale deep network models. The resulting “pretrained” networks can be further trained on a new dataset of interest, often outperforming models prepared from scratch, only on the target dataset. For brain-computer interface-based (BCI) neuroprosthetics, leveraging such pretrained models could potentially yield higher performance, increasing system reliability or reducing user burden. We thus assess the value of pretrained networks for BCI, namely for decoding neural activity recorded from the motor cortex.

Methods: We developed Neural Data Transformer 3 (NDT3), a deep network model that converts intracortical spiking activity into upper limb control signals, such as reaching kinematics or muscle activity. NDT3 was pretrained on 2000 hours of neural and behavioral data collected from motor neuroscience studies involving over 30 subjects, including both nonhuman primates and human participants enrolled in clinical BCI trials. We evaluated NDT3 in both offline decoding and online control settings. In offline decoding, we compare a pretrained NDT3 model against decoders trained from scratch on eight diverse motor datasets. For online control, we evaluate NDT3 against an optimal linear estimator (OLE)[1] for the use of a computer cursor in three human participants.

Results: For offline decoding, NDT3 provides a drop-in replacement that outperforms existing decoders when modeling datasets shorter than 1.5 hours. Beyond this threshold, NDT3 matches the performance of models trained from scratch. Moreover, pretrained NDT3 maintains its performance advantage beyond the specific setting it is tuned for. For example, while NDT3 and linear models both see performance declines on data collected an hour after an initial training dataset, NDT3 maintains a consistent margin above the linear model.

In closed loop control experiments, the NDT3 model provides higher performance in 2D center-out target acquisition than OLE controllers when both are trained from an open-loop calibration block. This is true across participants who have implants of varying signal qualities. NDT3 produces similar trajectories to OLE in high quality regimes, but prefers slower trajectories to enable better control in lower quality regimes.

Conclusion: Our results show that large-scale deep networks enable high performance decoding on varied intracortical motor data. Such models can be used to analyze new motor datasets offline, and more tentatively, may provide practical advances for neural prosthetics.