Our research focuses on three key areas: speech perception, speech production, and brain-computer interfaces (BCIs) for restoring communication. We study how the brain understands speech sounds and how different brain regions control the muscles needed for speaking. Our ongoing BCI clinical trial is testing new ways to make the technology faster, more accurate, and capable of working in multiple languages, helping people regain their ability to communicate naturally. Additionally, we are involved in clinical trials exploring new treatments for depression and epilepsy.
Discovering the Building Blocks of Speech
Speech is made up of basic sounds called phonemes (like the sounds /b/, /d/, or /a/). These building blocks combine to form the words and sentences we use every day. Our lab made significant discoveries about how the STG works to process these sounds, including:
2010: We found that the STG recognizes phoneme categories, even when their sounds vary (e.g., different speakers or accents) [1]
2014: We discovered that the brain doesn’t just process individual phonemes. Instead, it focuses on smaller, universal speech features shared across languages [2].
2016: We found that the STG can "fill in" missing sounds when speech is unclear, possibly working with other parts of the brain to help us understand what’s being said [3].
2024: Using cutting-edge NeuroPixels technology, we’ve identified how individual brain cells respond to specific features of speech sounds, like consonants and vowels [4].
These insights bring us closer to understanding how the brain transforms sounds into meaning and helps us explore potential therapies for speech and language disorders.
References
[1] Categorical speech representation in human superior temporal gyrus. Chang, E., Rieger, J., Johnson, K. et al., Nat Neurosci 13, 1428–1432 (2010). https://doi.org/10.1038/nn.2641
[2] Phonetic feature encoding in human superior temporal gyrus. Mesgarani N, Cheung C, Johnson K, Chang EF., Science. 2014 Feb 28;343(6174):1006-10. https://doi.org/10.1126/science.1245994
[3] Perceptual restoration of masked speech in human cortex. Leonard, M., Baud, M., Sjerps, M. et al. Nat Commun 7, 13619 (2016). https://doi.org/10.1038/ncomms13619
[4] Large-scale single-neuron speech sound encoding across the depth of human cortex. Leonard MK, Gwilliams L, Sellers KK, Chung JE, Xu D, Mischler G, Mesgarani N, Welkenhuysen M, Dutta B, Chang EF., Nature. 2024 Feb;626(7999):593-602. https://doi.org/10.1038/s41586-023-06839-2
Understanding How We Hear and Process Speech
Our research explores how the brain makes sense of the words we hear every day. Some of the big questions we’re investigating include:
How does the brain figure out where one word ends and another begins?
What parts of the brain help us understand the meaning of speech?
By answering these questions, we aim to uncover how our brains process language and what happens when this ability is disrupted, such as in certain neurological conditions.
How the Brain Processes Speech
A key focus of our research is understanding how the brain processes the sounds we hear when someone speaks. We study a part of the brain called the superior temporal gyrus (STG), which plays a critical role in recognizing and interpreting speech sounds.
By analyzing how the brain responds to natural speech, we’ve learned how different elements of speech, like individual sounds, syllables, and words, are represented in the brain.
How We Speak: The Brain and Speech Production
Speaking is an incredible process that involves the coordination of over 100 muscles in the vocal tract, including the tongue, lips, and vocal cords. Our research seeks to understand how the brain controls these muscles to produce clear and fluent speech.
Some key questions we explore include:
How do we produce sounds in the correct sequence to form words and sentences?
How does the brain prepare and plan speech before we start talking?
What happens in the brain when we stop speaking?
Insights from Our Research
Our research has shown that the ventral sensorimotor cortex (vSMC) plays a vital role in controlling the movements of the vocal tract, while also uncovering the contributions of other regions, such as the middle precentral gyrus (midPrCG), to speech production. Key findings include:
2013: Revealed how the brain encodes precise movements required for speech, such as shaping the mouth and moving the tongue, beyond its focus on linguistic units like phonemes. [1]
2018: Identified how the brain encodes movement parameters of the vocal tract that enable speech production. [2-3]
2022: Highlighted the broader role of the middle precentral gyrus (midPrCG), traditionally associated with speech muscle control, through neurosurgical studies demonstrating its significance in speech production. [4]
While we now understand more about how the brain directly controls speech movements, this raises new questions:
How do different brain regions work together to plan and coordinate speech?
How are motor plans for speech created and executed?
Our work continues to explore these fascinating questions, helping us uncover the mysteries of how we turn thoughts into words.
References
[1] Functional organization of human sensorimotor cortex for speech articulation. Bouchard KE, Mesgarani N, Johnson K, Chang EF., Nature. 2013 Mar 21;495(7441):327-32. https://doi.org/10.1038/nature11911
[2] Encoding of articulatory kinematic trajectories in human speech sensorimotor cortex. Chartier J, Anumanchipalli GK, Johnson K, Chang EF., Neuron. 2018 Jun 6;98(5):1042-1054.e4. http://doi.org/10.1016/j.neuron.2018.04.031
[3] Human sensorimotor cortex control of directly measured vocal tract movements during vowel production. Conant, David F. and Bouchard, Kristofer E. and Leonard, Matthew K. and Chang, Edward F., J. Neurosci. 2018 Mar 21, 38 (12) 2955-2966; https://doi.org/10.1523/JNEUROSCI.2382-17.2018
[4] A neurosurgical functional dissection of the middle precentral gyrus during speech production. Silva AB, Liu JR, Zhao L, Levy DF, Scott TL, Chang EF., J Neurosci. 2022 Nov 9;42(45):8416-8426. https://doi.org/10.1523/JNEUROSCI.1614-22.2022
Image created for the publication of “A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages” by Ella Maru Studio.
References
The speech neuroprosthesis. Silva AB, Littlejohn KT, Liu JR, Moses DA, Chang EF. Nat Rev Neurosci. 2024 Jul;25(7):473-492. https://www.nature.com/articles/s41583-024-00819-9
Neuroprosthesis for decoding speech in a paralyzed person with anarthria. Moses, D. A., Metzger, S.L., Liu, J.R., et al., N Engl J Med 385, 217–227 (2021). https://www.nejm.org/doi/full/10.1056/NEJMoa2027540
Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis. Metzger, S. L., Liu, J. R., Moses, D. A., et al., Nature Communications, 13(1), 6510 (2022). https://www.nature.com/articles/s41467-022-33611-3
A high-performance neuroprosthesis for speech decoding and avatar control. Metzger, S. L., Littlejohn, K. T., Silva, A. B., Moses, D. A., Seaton, M. P., et al., Nature, 620, 1037–1046 (2023). https://www.nature.com/articles/s41586-023-06443-4
A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages. Silva, A.B., Liu, J.R., Metzger, S.L. et al., Nat. Biomed. Eng 8, 977–991 (2024). https://doi.org/10.1038/s41551-024-01207-5
Restoring Communication with Brain-Computer Interfaces (BCIs)
For people with severe paralysis caused by conditions like ALS or brainstem strokes, losing the ability to speak can deeply impact their ability to connect with others and express themselves. While some use communication devices, these are often slow and challenging. However, many individuals still have the ability to form words and thoughts in their minds, even if they cannot speak them aloud.
Our lab is pioneering a technology called Brain-Computer Interfaces (BCIs) that can restore natural communication by translating brain signals directly into speech [1]. These devices work by detecting brain activity related to speech and converting it into text, sound, or even facial expressions on a virtual avatar.
Since 2018, our clinical trials have achieved the following milestones:
2021: Decoded a small vocabulary of 50 words into text on a screen [2].
2022: Expanded this to a spelling system that allows for over 1,000 words [3].
2023: Successfully translated continuous speech into text, audio, and an animated avatar that resembles the user [4].
2024: Began decoding multilingual speech, including both English and Spanish [5].
Today, we’re working to make this technology even faster, more accurate, and capable of handling larger vocabularies. We’re also exploring how it can work in more languages and creating lifelike avatars for a more immersive experience. Our goal is to bring back natural and effortless communication for people who have lost their voice.
Ethics
In our neurosurgery lab, ethics guide everything we do. We are committed to two core principles: protecting patient care and ensuring research participation is always voluntary [1-2]. Medical decisions are made solely in the patient’s best interest, never influenced by research. We take extra steps to ensure patients fully understand their choices and never feel pressured. Our goal is to advance science while always prioritizing trust, transparency, and patient well-being.
References
Ethical commitments, principles, and practices guiding intracranial neuroscientific research in humans. Feinsinger A, Pouratian N, Ebadi H, Adolphs R, Andersen R, Beauchamp MS, Chang EF, Crone NE, Collinger JL, Fried I, Mamelak A, Richardson M, Rutishauser U, Sheth SA, Suthana N, Tandon N, Yoshor D; NIH Research Opportunities in Humans Consortium. Neuron. 2022 Jan 19;110(2):188-194.
Neurosurgical Patients as Human Research Subjects: Ethical Considerations in Intracranial Electrophysiology Research. Chiong W, Leonard MK, Chang EF. Neurosurgery. 2018 Jul 1;83(1):29-37.