Latest prosthetic converts brain signals to speech in real time

A speech prosthetic developed by a collaborative team of Duke neuroscientists, neurosurgeons, and engineers can translate an individual’s brain signals into what they’re attempting to say.

Appearing Nov. 6 within the journal Nature Communications, the brand new technology might in the future help people unable to speak attributable to neurological disorders regain the power to speak through a brain-computer interface.

“There are numerous patients that suffer from debilitating motor disorders, like ALS (amyotrophic lateral sclerosis) or locked-in syndrome, that may impair their ability to talk,” said Gregory Cogan, Ph.D., a professor of neurology at Duke University’s School of Medicine and considered one of the lead researchers involved within the project. “But the present tools available to permit them to speak are generally very slow and cumbersome.”

Imagine listening to an audiobook at half-speed. That is the very best speech decoding rate currently available, which clocks in at about 78 words per minute. People, nonetheless, speak around 150 words per minute.

The lag between spoken and decoded speech rates is partially due the relatively few brain activity sensors that might be fused onto a paper-thin piece of fabric that lays atop the surface of the brain. Fewer sensors provide less decipherable information to decode.

To enhance on past limitations, Cogan teamed up with fellow Duke Institute for Brain Sciences faculty member Jonathan Viventi, Ph.D., whose biomedical engineering lab makes a speciality of making high-density, ultra-thin, and versatile brain sensors.

For this project, Viventi and his team packed a formidable 256 microscopic brain sensors onto a postage stamp-sized piece of flexible, medical-grade plastic. Neurons only a grain of sand apart can have wildly different activity patterns when coordinating speech, so it’s obligatory to differentiate signals from neighboring brain cells to assist make accurate predictions about intended speech.

After fabricating the brand new implant, Cogan and Viventi teamed up with several Duke University Hospital neurosurgeons, including Derek Southwell, M.D., Ph.D., Nandan Lad, M.D., Ph.D., and Allan Friedman, M.D., who helped recruit 4 patients to check the implants. The experiment required the researchers to position the device temporarily in patients who were undergoing brain surgery for another condition, comparable to treating Parkinson’s disease or having a tumor removed. Time was limited for Cogan and his team to check drive their device within the OR.

I like to check it to a NASCAR pit crew. We don’t need so as to add any overtime to the operating procedure, so we needed to be out and in inside quarter-hour. As soon because the surgeon and the medical team said ‘Go!’ we rushed into motion and the patient performed the duty.”

Gregory Cogan, Ph.D., professor of neurology, Duke University’s School of Medicine

The duty was a straightforward listen-and-repeat activity. Participants heard a series of nonsense words, like “ava,” “kug,” or “vip,” after which spoke each aloud. The device recorded activity from each patient’s speech motor cortex because it coordinated nearly 100 muscles that move the lips, tongue, jaw, and larynx.

Afterwards, Suseendrakumar Duraivel, the primary writer of the brand new report and a biomedical engineering graduate student at Duke, took the neural and speech data from the surgery suite and fed it right into a machine learning algorithm to see how accurately it could predict what sound was being made, based only on the brain activity recordings.

For some sounds and participants, like /g/ within the word “gak,” the decoder got it right 84% of the time when it was the primary sound in a string of three that made up a given nonsense word.

Accuracy dropped, though, because the decoder parsed out sounds in the center or at the tip of a nonsense word. It also struggled if two sounds were similar, like /p/ and /b/.

Overall, the decoder was accurate 40% of the time. That will look like a humble test rating, nevertheless it was quite impressive on condition that similar brain-to-speech technical feats require hours or days-worth of information to attract from. The speech decoding algorithm Duraivel used, nonetheless, was working with only 90 seconds of spoken data from the 15-minute test.

Duraivel and his mentors are enthusiastic about making a cordless version of the device with a recent $2.4M grant from the National Institutes of Health.

“We’re now developing the identical form of recording devices, but with none wires,” Cogan said. “You’d give you the chance to maneuver around, and you would not need to be tied to an electrical outlet, which is basically exciting.”

While their work is encouraging, there’s still an extended strategy to go for Viventi and Cogan’s speech prosthetic to hit the shelves anytime soon.

“We’re at the purpose where it’s still much slower than natural speech,” Viventi said in a recent Duke Magazine piece concerning the technology, “but you may see the trajectory where you would possibly give you the chance to get there.”

This work was supported by grants from the National Institutes for Health (R01DC019498, UL1TR002553), Department of Defense (W81XWH-21-0538), Klingenstein-Simons Foundation, and an Incubator Award from the Duke Institute for Brain Sciences.