Ann Johnson, who became paralyzed and lost her speech ability after a stroke 18 years ago, can now communicate verbally through a digital avatar thanks to a brain implant and artificial intelligence (AI). A study published in the journal Nature reveals that researchers implanted an array of electrodes on the surface of Ann’s brain. These transmit her brain activity to computers, where AI algorithms translate the signals into words. After a short delay, an on-screen avatar speaks these words out loud and reflects her emotion with facial expressions.
Edward Chang, a co-author of the study and a neurosurgeon at the University of California, San Francisco (UCSF), expressed satisfaction at the working of the study in real-time [NBC News]. On the other hand, neuroscientist Nick Ramsey from the University of Utrecht, who is not a contributor to the study, said the results represent a significant advancement from previous outcomes, indicating a “tipping point” [The Guardian].
Ann utilizes a device that translates head movements into typed words, allowing a communication rate of 14 words per minute (WPM), as per UCSF. A new interface, currently only for study use, has increased her WPM to 78 with about 75% accuracy. This is a significant improvement over a prior version from the same team, which had a rate of 15 WPM.
The improved communication system uses an implant with 253 electrodes placed on pivotal parts of the brain linked to communication. These electrodes detect signals that were previously sent to muscles related to speech before Ann’s stroke. The signals are transported to computers through a cable connected to a port on Ann’s head.
Then, AI breaks down these signals and translates them into individual sounds or phonemes, which are subsequently combined into words. A digital avatar resembling Ann delivers these words, with a voice trained to mimic hers using her wedding video clips. The avatar also reflects her emotions based on her brain signals.
[sourcelink link=”https://engineering.berkeley.edu/news/2023/08/novel-brain-implant-helps-paralyzed-woman-speak-using-a-digital-avatar/”]
