InnerVoice
Decoding imagined speech from the brain
Brain-to-speech decoding could provide a crucial lifeline for those who have lost the ability to communicate because of motor neuron disorders, spinal cord injury or stroke.
Large parts of the brain may still be functioning normally in people who are in a locked-in state. This presents an important opportunity to transform their neural data into intelligible signals for communication via a brain-computer interface (BCI). Previous attempts to re-enable communication have used implanted electrodes to achieve ‘yes/no’ responses, spell words letter by letter, control computer cursors, and to decode the imagined movements of handwriting.
Groups around the world are now exploring the possibility of decoding attempted speech into text to restore faster, more natural, communication.
Dr Silvia Marchesotti, from the University of Geneva, established closed-loop BCI experiments at the presurgical unit of Geneva University Hospital (HUG), as part of a study within the NCCR Evolving Language, a Swiss National Center exploring the origins and future of language, while research led by Dr Timothée Proix, under the direction of Professor Anne-Lise Giraud, at the University of Geneva and Professor Pierre Mégevand at the University of Geneva and HUG, identified neural features suitable for BCI decoding, laying important groundwork for this study.
Decoding words from participants implanted with neural electrodes.
Working closely with clinicians in the epilepsy monitoring unit at HUG, and with researchers at the University of Geneva Human Neuron Lab, and the Speech and Language Group, the Wyss Center team is decoding individual words from a large group of study participants implanted with a new type of ‘macro-micro’ electrode as well as with standard monitoring electrodes. The project takes place within the context of MicroEPI, a clinical trial that focuses on the new hybrid electrodes to reveal the pathophysiology of epilepsy. The Wyss Center team assisted Prof. Mégevand in the preparation of the technical documentation for the clinical trial approval, in collaboration with the electrode manufacturer, and is now performing the clinical monitoring.
The participants in this study are in hospital for epilepsy monitoring. This means that they already need to have neural electrodes implanted to help with their diagnosis, care, and treatment.
“The study participants are contributing to future assistive devices for people who can no longer communicate, but they will also help identify new biomarkers for epilepsy seizure detection and prediction at the neuronal level.”
Unlike other studies that have primarily placed electrodes in the brain’s motor cortex – the area responsible for movement –, this study will explore electrode placement in other brain regions with the potential to discover new targets for language decoding.
The novel design of the electrodes allows simultaneous measurement of action potentials (spikes) from individual neurons firing and the activity of multiple neurons working together (known as local field potentials or LFPs). This combination of signal types may lead to new paths for reliable speech decoding and help reveal brain states – which could identify when someone is ready to communicate.
“BCI studies are challenging so new approaches are often trialed in just one or two people. Importantly in this study, we are collecting data from around 30 people which will enable our algorithms to recognize typical variability in brain signals, improving stability and ultimately creating more usable software for patients and carers at home.”
The brain signals when we talk to ourselves
The participants are asked to imagine certain words while their brain response is recorded. They repeat the process multiple times so that a typical ‘brain signature’ is registered for that word. This signature is used to train a word recognition algorithm embedded in the Wyss Center’s NeuroKey software platform.
Speech therapists have advised the team on which words to select for the study. Words that could actively help people with paralysis in their daily life are prioritized. As the majority of participants are native French speakers, words such as ‘soif’ (thirsty), ‘faim’ (hungry), ‘chaud’ (hot), froid (cold) are included.
In addition to advancing brain-speech decoding for communication, the results of the study will help assess the type of electrodes that have the potential to deliver future performance for chronic BCI use.
The InnerVoice project is part of the Wyss Center’s BCI program and interacts closely with INTRECOM, the comprehensive BCI communication system for patients in locked-in state. It is anticipated that the technology and algorithms developed in InnerVoice will be used in a later stage clinical study with the Wyss Center’s ABILITY system.
