(28) Cross-patient Speech Decoding from Shared Latent Representations of Speech Production

Introduction: Speech brain-computer interfaces (BCIs) show promise for restoring verbal communication to individuals with an inability to speak due to neurodegenerative disorders. However, state-of-the-art speech BCIs are limited by long training times because they employ complex and patient-specific machine learning models that demand weeks of data collection from a single patient for effective training. Recent work has shown cortical neural signals are preserved across patients when performing simple motor tasks, suggesting that smaller patient-specific datasets could be pooled across multiple patients to accelerate model training. We hypothesize that there is a shared, cross-patient representation of speech production that will enable speech BCIs to train on neural data from multiple patients, reducing the quantity of individual patient data needed and saving training time.

Methods: We placed high-density micro-electrocorticographic arrays over the sensorimotor cortex (SMC) of eight intraoperative patients instructed to perform a speech repetition task. We extracted time-varying high-gamma power (HG: 70-150 Hz) from cortical signals recorded during the patients’ responses. We constructed latent feature spaces by decomposing HG dynamics with principal component analysis and subsequently aligned these spaces across patients to a common neural latent space using canonical correlation analysis (CCA). To investigate the speech content encoded in these shared dynamics, we trained support vector machine decoding models to predict the constituent phonemes of patients’ responses from cross-patient latent dynamics.

Results: We found that CCA alignment successfully finds affine transformations of patients’ latent dynamics that increase their correlations across patients, suggesting these latent representations of speech production are shared between patients. These aligned latent dynamics retained encoded speech information, as decoding models predicted phonemes with higher accuracy when trained on cross-patient data aligned to a common latent space (31.3 ± 13.0%, chance: 11.1%) than when trained on patient-specific data (24.1 ± 10.6% chance: 11.1%). Further, this alignment technique allowed models to predict phonemes with accuracies well above chance on data from patients they had not been trained on (30.3 ± 12.1%, chance: 11.1%). These results suggest that there is a shared neural representation of speech production in the SMC across patients which can enable speech decoding models to successfully train on cross-patient data.

Conclusion: The next generation of speech BCIs can use this shared neural representation of speech from multiple patients to reduce training time and learn a robust, cross-patient representation of speech production.