Mind-Reading Tech Could Bring 'Synthetic Speech' to Brain-Damaged Patients

Media playback is unsupported on your device                  Media caption Listen to speech decoded from brain activity

Media playback is unsupported on your device Media caption Listen to speech decoded from brain activity

WEDNESDAY, April 24, 2019 (HealthDay News) - Reading the brain waves that control a person's vocal tract might be the best way to help return a voice to people who've lost their ability to speak, a new study suggests. Some people with severe speech disabilities learn to spell out their thoughts letter-by-letter using assistive devices that track very small eye or facial muscle movements. Natural speech, by comparison, can reach up to 150 words per minute. The decoder was able to produce speech from this participant's brain activity even though the participant never produced any sound. It also hasn't yet been tested in patients who don't have the ability to speak.

An array of intra-cranial electrodes of the type used to record brain activity in the current study.

"We want to create technologies that can reproduce speech directly from human brain activity", he continued.

"The brain translates those thoughts into movements of the vocal tract, and that's what we're trying to decode".

"Clearly, there is more work to get this to be more natural and intelligible but we were very impressed by how much can be decoded from brain activity".

Although the speech sounds produced by were sometimes fuzzy, they were much more accurate than anything that's been created before. "We reasoned that if these speech centers in the brain are encoding movements rather than sounds, we should try to do the same in decoding those signals". To develop the interface, Chang and colleagues worked with five volunteers with intact speech who had electrodes temporarily implanted for intracranial monitoring during epilepsy surgery.

Audio recordings of the participants' voices were used to reverse-engineer the vocal tract movements required to make those sounds, including lip movements, vocal cord tightening, tongue manipulation and jaw movement. This detailed computer simulation of the anatomy used to create speech could then be controlled by brain activity.

With the development of a decoder that translates brain signals into synthetic speech, a team of neuroscientists has now taken crucial steps toward someday restoring the voices of people who have lost the ability to speak.

The scientists wrote: "Listeners were able to transcribe synthesised speech well".

For most people, speech is their main form of communication and a way of expressing how they feel. For example, they accurately identified 69% of synthesized words from lists of 25 alternatives, including 43% of sentences transcribed with ideal accuracy.

By breaking down the problem of speech synthesis into two parts, the researchers appear to have made it easier to apply their findings to multiple individuals.

Still, he added, "The levels of accuracy we produced here would be an awesome improvement in real-time communication compared to what's now available".

The researchers are now experimenting with higher-density electrode arrays and more advanced machine learning algorithms that they hope will improve the synthesized speech even further.

Josh Chartier, a co-author of the new study, maintains that the levels of accuracy produced by their system improves upon existing technologies, but acknowledges that there is a ways to go to be able to perfectly mimic spoken language.

Preliminary results from one of the team's research participants suggest that the researchers' anatomically based system can decode and synthesize novel sentences from participants' brain activity almost as well as the sentences the algorithm was trained on.

The researchers realised that a virtual vocal tract was necessary to translate the signals.

NFL draft: Biggest surprises of the 1st round
Two injured in Port Talbot steelworks explosion