UCSF Researchers Publish Avatar System Restoring Speech, Gestures in Nature Neuroscience

A brain-computer interface linked to a virtual avatar let three people with severe paralysis express both speech and gestures at the same time, researchers at the University of California, San Francisco reported on September 14, 2026.

The system decoded neural signals from a single implanted electrode array and translated them into commands that drove a personalized full-body avatar. In tests, the device reached up to 82% accuracy for gestures alone and 65%–70% for simultaneous speech and gestures, while conversational accuracy hit at least 85% for gestures and 75% for speech, according to findings published in Nature Neuroscience.

One implant captures two channels of expression

Paralysis from stroke or neurodegenerative disease often strips away both voice and movement, leaving people unable to speak or gesture — the twin pillars of natural conversation. Earlier brain-computer interfaces (BCIs) tackled one or the other. The UCSF team, led by Dr. Edward Chang and funded by the National Institutes of Health, showed that a single high-density electrocorticography (ECoG) implant over the motor cortex can capture signals for both.

Three participants with severe vocal-tract paralysis took part. Two also had severe limb paralysis; the third had moderate limb paralysis. Each was connected to a virtual avatar that mirrored upper-body movements such as a hand wave or thumbs-up alongside facial expressions synced to decoded speech.

“Conversation is about much more than the words being spoken,” Chang said. “It's a multilayered, dynamic process involving the whole motor cortex. This proof-of-concept shows us it's possible for a BCI to restore some of this freedom and flexibility.”

Training on both modes boosts performance

The researchers compared two training strategies: one using data from speech and gestures attempted simultaneously, the other using data from each mode in isolation. Training on the combined dataset improved decoding accuracy for both speech and gestures, suggesting the system benefits from learning the natural interplay between the two.

Two participants used the avatar to copy specific actions and respond to conversational prompts. The results move us closer to BCIs that restore the multimodal richness of face-to-face interaction, not just text or synthetic voice output.

Parallel advances in silent-speech control

Separate work published in Communications Biology on November 11, 2026, demonstrated a different BCI approach: a clinical-trial participant with amyotrophic lateral sclerosis (ALS) used a chronically implanted ECoG array to issue self-paced silent-speech commands that controlled smart-home devices. Across 18 sessions, silently mimed commands were decoded with a median accuracy of 97.1% and a median rate of 11.86 correct decodes per minute — no external timing cues required.

That study, led by researchers at Johns Hopkins Hospital and supported by the National Institute of Neurological Disorders and Stroke, shows that attempted speech alone can drive real-world device control autonomously and privately.

Long-term independent use moves closer

A third study, published in Nature Medicine, reported that a participant with tetraplegia operated an intracortical BCI for speech and cursor control independently at home for more than 19 months. The system maintained over 99% word accuracy with a 125,000-word vocabulary and required only minutes of daily calibration, even after hundreds of days post-implant.

From lab to life: what remains

Despite the progress, all three systems still rely on wired connections and external hardware. Chang’s team plans to transition to a fully implantable, wireless version. Safety, long-term reliability across diverse users, and real-world performance in unscripted conversation remain active research questions.

The global BCI market, valued at $1.7 billion in 2022, is projected to reach $6.2 billion by 2030, reflecting growing clinical momentum. But for the people these technologies aim to serve, the measure of success is simpler: the ability to raise a hand, speak a sentence, and be understood — at the same time.

Study details and author contributions

The Nature Neuroscience study was authored by Brosler SC, Liu JR, Silva AB, Hallinan IP, Kurtz-Miott CM, Wilker JFPD, Tu-Chan A, Ganguly K, and Chang EF, and published on September 14, 2026 (doi: 10.1038/s41593-026-02446-2). The work was funded by the NIH National Institute on Deafness and Other Communication Disorders (NIDCD).

The Communications Biology study was authored by Luo S, Angrick M, Coogan C, and colleagues, and published on November 11, 2026 (doi: 10.1038/s42003-026-10477-5). The authors were supported by the National Institutes of Health under award number UHSNS114439 (NINDS). The authors declared no competing interests.

BrainGate2 trial enrolls participants for further study

These technologies remain largely experimental but show promise for restoring functions lost due to ALS, stroke, and other neurological disorders. The BrainGate2 clinical trial is enrolling participants. Individuals interested in learning more about the study can visit https://www.braingate.org/.

The University of California, San Francisco traces its origins to Toland Medical College, founded in 1864, and became the University of California’s medical department in 1873. It gained full campus independence in 1964 and is dedicated entirely to health and life sciences. In 2023, UCSF received the second-highest research funding from the National Institutes of Health, and in 2021 spent $1.71 billion on research and development, the second most among U.S. higher-education institutions.

Readers should consult qualified medical professionals for guidance on clinical trial participation or treatment options. The findings discussed represent early-stage research and do not constitute medical advice.