BrainGate2 clinical trial participant T7 used the brain-computer interface to control a specially designed radial keyboard. Innovations in the system’s decoder keep the system calibrated automatically.
Credit: Braingate.org
PROVIDENCE, R.I. [Brown University] — A new study in Science Translational Medicinedescribes three software innovations that substantially improved the user experience and performance of the BrainGate* brain computer interface (BCI). Researchers said the gains are a significant advance in their ongoing work to develop and test a practical BCI assistive technology that people with paralysis could use easily, reliably, independently, and on demand to regain control over external devices.
Intracortical BCIs such as BrainGate use a tiny array of implanted electrodes to pick up the electrical activity of neurons in the motor cortex of the brain. Computers then translate those signals into digital commands that have allowed users to control electronic devicessuch as computers and robotic arms by simply intending to move their own arm or hand. The translation relies on a decoder, an algorithm that infers the movement intentions of the user from the patterns of their neural activity.
A challenge of decoding movement intention from intracortical electrodes is that the signals change over time — thus, intracortical BCIs have required frequent interruptions for decoder recalibration. Neural signal patterns can change when the electrodes move even slightly; a neuron whose signal was not previously detected can end up joining the recorded ensemble while another might become excluded. As the neural signals shift and drift, the performance of the BCI — the ability of users to move a cursor or robot by thinking about the move — will decline until the decoder can be recalibrated. This recalibration is typically performed using a special task in which the participant tries to move the cursor to prescribed targets so that movement intentions can be mapped to the new neural activity patterns.
The essential advance in the new study is a set of decoder upgrades that allow the algorithm to recalibrate itself during practical BCI use without making the user stop for calibration task every time the signals change. Results of the research reported in the paper show that the new decoder preserved BCI performance much longer than before and even contributed to improving users' accurate typing speed on an on-screen keyboard. Rather than frequent pauses for recalibration, users could type for hours, pausing only when they wanted to and without the need for technicians to intervene.
"Eliminating the need to run a calibration task whenever the recorded signals change will make a clinical BCI more user-friendly and easy to use," said lead author Beata Jarosiewicz, assistant professor (research) of neuroscience at Brown University and the Brown Institute for Brain Science, and investigator at the Providence Veterans Affairs Medical Center (PVAMC).
The BrainGate team includes scientists, engineers, and clinicians from Brown, Massachusetts General Hospital (MGH), PVAMC, Stanford University, and Case Western Reserve University.
In a particularly dramatic demonstration at the Stanford site of the pilot clinical trial, a woman identified as "T6," who is diagnosed with amyotrophic lateral sclerosis, was able to use BrainGate for six sessions of a few hours each over the course of 42 days without any interruptions for explicit recalibration after the decoder was initialized on the first day. T6 was able to type paragraphs, pausing and unpausing the BCI on her own, while the decoder calibrated itself.
"Watching our participants use this more robust system to type on a computer screen highlights the progress being made toward a clinically useful system," said Dr. Leigh Hochberg, professor of engineering at Brown, director of the Center for Neurorestoration and Neurotechnology at PVAMC, director of the Neurotechnology Trials Unit at MGH Neurology, and senior author of the paper.
Clever coding
The innovations take advantage of the information that accrues as trial participants use the system. In one innovation, dubbed "retrospective target inference," the decoder analyzes the users' recent target selections to update its interpretation of the underlying neural signals that produced it. For example, if a participant is moving the cursor around to type "B-R-A-I-N-G-A-T-E," Jarosiewicz said, the decoder has definitive new information about the user's movement intentions during the trajectory between each letter. For example, when the cursor was moving from B to R, the person's movement intentions can be inferred to have been directly toward the R.
"We can use that information to update the map between the neural signals and the decoder's 'understanding' of the person's movement intentions," Jarosiewicz said.
Meanwhile, the decoder now also keeps track of the baseline levels of neural activity in the motor cortex during moments when participants have chosen to pause. Because of the changes in neural signals over time, the resting firing patterns of neurons in the motor cortex might have changed since the last pause. Keeping track of this baseline rate during pauses helps the system to start with better calibration when the user decides to turn the system back on.
The third innovation is keeping track, while participants use the system, of emerging biases in the velocity of cursor movement and subtracting them out of the decoded movements. Keeping a better balance of speeds in different directions leads to more accuracy and a more intuitive user experience.
The team performed several research sessions to confirm that the new innovations were making a significant and positive difference among the participants. These included asking participants to type for an hour or two in some sessions when the self-calibration innovations were engaged and some when they were turned off. Performance remained stable in sessions when the features were on, but degraded significantly when the features were off.
In one of the research sessions with the innovations turned off, the researchers found that turning them back on allowed the system to rescue performance after it had been degrading for lack of the features.
In principle, the improved decoder can be applied to other BCI tasks such as three-dimensional control of a robot arm or even a person's own electronically reanimated arm or hand. There are unique challenges to doing that, Jarosiewicz acknowledged, but it's a problem the team is tackling next.
The goal remains to develop BrainGate into a system that would be available to people who need it, whenever they need it.
"There is still a lot of research to do," Hochberg said. "With continued clinical research, we will learn how our findings extend to more participants. We want to make the system faster, easier, smaller, fully implanted, more portable, less requiring of an expert researcher or caregiver, and more nimble in its ability to provide control of external devices.
"In these studies, we are making steps toward robust and flexible communication systems for people with severely limited movement, including limited or no speech. We are also dedicated not only to enabling control over computers or robotic assistive devices, but — for people with spinal cord injury or stroke — working toward the goal of reconnecting brain to limb, allowing the powerful intracortical signals to activate fully implanted functional electrical stimulation devices, and re-enabling intuitive movement of one's own arm and hand."
Hochberg emphasized, "Our extraordinary research participants are true pioneers. They are participating in the trial not because they hope to gain any personal benefit, but because they want to help us to develop and test a system that will help other people with paralysis in the future."
In addition to Jarosiewicz and Hochberg, the paper's other authors are Anish Sarma, Daniel Bacher, Nicholas Masse, John Simeral, Brittany Sorice, Erin Oakley, Christine Blabe, Chethan Pandarinath, Vikash Gilja, Sydney Cash, Emad Eskandar, Gerhard Friehs, Jaimie Henderson, Krishna Shenoy, and John Donoghue.
The research was supported by the Department of Veterans Affairs (Rehabilitation Research & Development Service awards: B6310W, B6453R, B6459L, A6779I); the National Institutes of Health (NIDCD grants R01DC009899, R01DC014034, NICHD-NCMRR grants N01HD10018, RC1HD063931, N01HD53403, and NINDS grant R01NS066311-S1); the Doris Duke Charitable Foundation; the MGH-Deane Institute; the Joseph Martin Prize for Basic Research; the Katie Samson Foundation; the Craig H. Neilsen Foundation; Stanford BioX-NeuroVentures; and the Garlick Family.
*Caution: Investigational device. Limited by federal law to investigational use.
By David Orenstein