Andrew Huberman interviews neurosurgeon Dr Eddie Chang on the science of speech, language, and brain-machine interface
Andrew Huberman speaks with Dr Eddie Chang, neurosurgeon and chair of neurological surgery at UCSF, about the neuroscience of speech and language.
Summary
Dr Eddie Chang, neurosurgeon and chair of neurological surgery at UCSF, joins Andrew Huberman to explore how the brain produces and processes speech and language. He explains the distinct neural and physiological mechanisms underlying speech production — from the larynx and vocal tract to the cortical areas that coordinate articulation — and distinguishes these from the broader cognitive systems that handle meaning, grammar, and comprehension. A significant portion of the conversation focuses on his clinical trial, the Bravo trial, in which electrodes implanted on the speech cortex of a paralyzed man allowed a machine learning algorithm to decode intended words from brain activity alone — the first time such a feat had been achieved. He also discusses the future of brain-machine interface technology, including avatar-based communication, the ethics of cognitive augmentation, and the neuroscience of stuttering.
Key Takeaways
FULL TRANSCRIPT
Introduction to Speech vs. Language
Andrew Huberman: Welcome to Huberman Lab Essentials, where we revisit past episodes for the most potent and actionable science-based tools for mental health, physical health, and performance. I'm Andrew Huberman and I'm a professor of neurobiology and ophthalmology at Stanford School of Medicine. And now for my discussion with Dr Eddie Chang. Eddie, welcome.
Dr Eddie Chang: Hi. Hi, Andrew.
Andrew Huberman: Great to be here with you. Your main focus these days is the neurobiology of speech and language. For those that aren't familiar, could you please distinguish for us speech versus language in terms of whether or not different brain areas control them? When I think about language, I think about words and just talking. If I sit down to do a long podcast or I think about asking you a question, I don't even think about the words I want to say very much — I have to think about them a little bit, one would hope — but I don't think about individual syllables unless I'm trying to accent something, or it's a word that I have a particular difficulty saying, or I want to change the cadence. So what in the world is contained in these brain areas? What is represented? To me that is perhaps one of the most interesting questions, and I know this lands square in your wheelhouse.
Dr Eddie Chang: Let's get into this, Andrew, because this is one of the most exciting areas right now — understanding how the brain processes these exact questions. Speech corresponds to the communication signal. It corresponds to me moving my mouth and my vocal tract to generate words, and you're hearing these as an auditory signal. Language is something much broader. It refers to what you're extracting from the words that I'm saying. We call that pragmatics — are you getting the gist of what I'm saying? There's another aspect we call semantics: do you understand the meaning of these words and sentences? There's another part we call syntax, which refers to how the words are assembled in a grammatical form. Those are all really critical parts of language, and speech is just one form of language. There are many other forms, like sign language and reading. Those are all important modalities.
Our research really focuses on this area that we're calling speech — the production of this audio signal, which you can't see, but your microphones are picking up. There are these vibrations in the air that are created by my vocal tract, picked up by the microphone in the case of this recording, but also picked up by the sensors in your ear. The very tiny vibrations in your ear are picking that up and translating it into electrical activity. It's such a complex feat. Some people would say it's the most complex motor thing that we do as a species — just speaking. Not the extreme feats of acrobatics or athleticism, but speaking.
Andrew Huberman: Especially when one observes opera, or freestyle rappers. And of course it's not just the lips — it's the tongue. You've mentioned two other structures: the pharynx and the larynx. Can you tell us at a superficial level what the pharynx and larynx do differentially? I think most people aren't going to be familiar.
How the Vocal Tract Produces Sound
Dr Eddie Chang: I'll talk primarily about the larynx here for a second. If you think about when we're speaking, really what we're doing is we're shaping the breath. Even before you get to the larynx, you've got to start with the expiration. We fill up our lungs and then we push the air out — that's a normal part of breathing. What is really amazing about speech and language is that we evolved to take advantage of that normal physiological process at the larynx. What the larynx does is that when you're exhaling, it brings the vocal folds together. Some people call them vocal cords, but they're not really cords — they're vocal folds. They're two pieces of tissue that come together, and a muscle brings them together. When the air comes through the vocal folds when they're together, they vibrate at really high frequencies, like 100 to 200 hertz. The reason why men and women generally have different voice qualities has to do with the size and shape of the larynx. In general, men have a larger voice box or larynx, and the resonance frequency of the vocal folds when the air comes through them is about 100 hertz for men and about 200 for women.
So you take a breath in. As the air is coming out, the vocal folds come together. The air goes through. That creates the sound of the voice — what we call voicing. It's not just your voice characteristic; it's the energy of your voice. It's coming from the larynx. It's a noise, the source of the voice. Then that energy, that sound, goes up through the parts of the vocal tract like the pharynx into the oral cavity — your mouth, your tongue, your lips. What those things are doing is shaping the air in particular ways that create consonants and vowels. That's what I mean by shaping the breath. It starts with this exhalation, you generate the voice in the larynx, and then everything above the larynx is moving around — just like the way my mouth is doing right now — to shape that air into particular patterns that you can hear as words.
Andrew Huberman: This immediately makes me wonder about more primitive or non-learned vocalisations, like crying or laughter. Are those produced by the language areas, or do they have their own unique neural structures?
Dr Eddie Chang: We call those vocalisations. A vocalisation is basically where someone creates a sound like a cry or a moan. It also involves the exhalation of air. It also involves some phonation at the level of the larynx where the vocal folds come together to create that audible sound. But it turns out those are actually different areas. People who have injuries in the speech and language areas can oftentimes still moan, still vocalise. It is a different part of the brain — an area that even non-human primates have, specialised for vocalisation. It's a different form of communication than words.
The Bravo Trial: Decoding Speech from a Paralysed Brain
Andrew Huberman: Speaking of the storage of and ability to speak, you are doing some amazing work and have achieved some pretty incredible, well-deserved recognition for your work in bringing language out of paralysed people — essentially allowing people who are locked in to a paralysed state, or otherwise unable to articulate speech, to communicate using brain-machine interface, translating the neural activity of areas of the brain that would produce speech into hardware and artificial, non-biological tools.
Dr Eddie Chang: There are a series of conditions — they include things like brain stem stroke. The brain stem is the part of the brain that connects the cerebrum, which is the top part that does our thinking and a lot of the motor control, speech, language, everything, to the spinal cord and the nerves that go out to the face and vocal tract. So if you have a stroke there, you could be thinking all the wild, creative, intelligent thoughts you have in the mind and the cerebrum, but you can't get them out into words, or you can't get them out to your hand to write them down. That's a very severe form of paralysis called brain stem stroke.
There's another kind of condition that we call neurodegenerative, where the nerve cells die or atrophy — a condition called ALS. That's a very severe form of paralysis. In its extreme form, people essentially lose all voluntary movement. The muscles to their diaphragm and their lungs essentially give out as well, and then they can't breathe anymore. In our field, these are the most devastating things that can happen. This condition of what we call being locked in refers to the idea that you can have completely intact cognition and awareness but have no way to express that — no voluntary movement, no ability to speak. That is devastating because psychologically and socially you're completely isolated. That's what we call locked-in syndrome, and it's devastating.
So we've been studying this patterning of electrical activity for consonants and vowels. Once we figured out a lot of these codes for the individual phonetic elements, part of the lab started to focus on this very specific question for people who have these kinds of paralysis: could we intercept those signals from the brain, the cerebral cortex, as someone is trying to say those words? And then can we have them taken out of the brain through wires to a computer that will interpret those signals and translate them into words?
We started a clinical trial called the Bravo trial. It's still underway. The first participant was a man who had been paralysed for 15 years. He was in a car accident. He actually walked out of the hospital the day after that car accident, but the next day had a complication where he had a very large stroke in the brain stem, and that turned out to be devastating. He didn't wake up from that stroke for about a week. When he woke up from that coma, he realised that he couldn't speak or move his arms or legs. As he communicated to us, that was absolutely devastating. He wanted really to die at that time.
Andrew Huberman: Could he blink his eyes or move his mouth in any way?
Dr Eddie Chang: He could blink his eyes. He had some limited mouth movements but couldn't produce any intelligible speech — it was completely slurred and incomprehensible. He survived this injury. A lot of people who have that kind of stroke just don't survive. The way he actually communicates, because he has a little bit of residual neck movement, is that he improvised and had his friends put a stick attached to his baseball cap, because he could move his neck. He would essentially type out letters on a keyboard screen to get out words — pecking out letters one by one by moving his neck to control this stick attached to his baseball cap. He hadn't really spoken for about 15 years.
Andrew Huberman: Oh, goodness.
Dr Eddie Chang: Yeah. So it was part of a clinical trial — something that our hospital and the FDA had to approve and looked at very carefully. But given a lot of the work that we had done, there was some basis for why this might work. We did a surgery where we implanted electrodes onto the areas that control the vocal tract — the areas that control the larynx, the areas that control the lips, tongue, and jaw movements when we normally speak. These are areas that presumably may be active. That was our hope. He underwent brain surgery. We put in an electrode array and connected it to a port that was screwed to his skull. The port actually goes through his scalp, and he's lived with this now for the last three years.
So he has an electrode array implanted over the part of the brain that's important for speech. It's connected to a port, and then we connect a wire to that port that translates what we call analog brain waves and converts them into digital signals. We put them through a machine learning or artificial intelligence algorithm that can pick up these very, very subtle patterns — you can't actually see them with your eye in the brain activity — and translate those into words. This took weeks to train the algorithm to interpret correctly.
What was incredible was to see how he reacted. He would be prompted to say a given word — like "outside," for example — and then he would think about it, try to say it, and finally those words would appear on the screen. You could really tell that he got a kick out of that, because his body would shake in a way, his head would shake in a way — he would start to giggle. That was cool to see, but then I also realised that when he was giggling, it kind of screwed up the next words being decoded.
Andrew Huberman: Is that a bug you've since fixed?
Dr Eddie Chang: No, we haven't fixed that. It's easier just to tell him to stop giggling.
The way this worked was we trained the computer to recognise 50 words. We started with a very small vocabulary that's expanding as we speak. I think it's just a matter of time before these vocabularies become much, much larger. We created essentially all the possible sentences that you could generate from those 50 words. Why that was important was that you can use all those possible sentences to create a computational model of all the different word combinations to give different sentences. And then you can essentially do what we call autocorrect — the same kind of thing that happens when you're texting. You get the wrong letter in there, your phone knows from context what to correct it to. Because the decoding is not 100% correct all the time — in fact, it's far from that — it's really helpful to have these other features like autocorrect that make it correct and then update it.
So it's a combination of a lot of things: the AI that is translating those brain activity patterns, but also things that we've learned from speech technologies that you put all together and all of a sudden it starts to work. That was the first time that someone who was paralysed could create words and sentences decoded from brain activity alone.
Brain-Machine Interface, Augmentation, and Ethics
Andrew Huberman: These days we hear a lot about Neuralink, Elon Musk's company. While brain-machine interface of the sort that you do has been going on for a long time, there's been some press around Neuralink about the promise of what brain-machine interface could do. What are your thoughts about manipulating neural circuitry to achieve superhuman or super-physiological functions?
Dr Eddie Chang: It's a really interesting time right now. The science has been going on for decades. The work that we've done in this field — brain-machine interface — has been going on for a while, and a lot of the early work was just trying to restore things like arm movement, or having people or monkeys control a computer cursor on a screen. That's been going on for decades. What's really new is that industry is now involved, and some of this is now becoming commercialised. We're starting to see us cross over to this field where it's no longer just research — we're talking about medical products that are designed to be surgically implanted. In some cases, people are doing this kind of work non-invasively as well, without surgery.
The specific question you were asking about is an area that we call augmentation. Can you build a device that essentially enhances someone's ability beyond normal — super memory, super communication speeds beyond speech, superior precision athletic abilities? I think these are very serious questions to be asking now, because the pathway so far is really to focus on medical applications. I personally don't think that we've thought enough about what these scenarios are going to look like, and I don't think we've thought through all the ethical implications of what this means for augmentation in particular.
There's part of this that is not new at all. Humans throughout history have been doing things to augment our function — coffee, nicotine, all kinds of medications that cross over from medical to consumer. That is everywhere. So the pursuit of augmentation or performance enhancement is really not a new thing. The questions as they relate to neurotechnologies have to do with the invasive nature. If these technologies require surgery for something that is not a medical application, that is not exactly new territory either — people do that routinely for cosmetic procedures for physical appearance. So I do think that, provided the technology continues to emerge the way that it does, it's going to be around the corner. And it probably is not going to be in ways that are super obvious. I don't think it's going to be like "can we easily memorise every fact in the world," but in forms that are going to be much more incremental and maybe more subtle.
In many ways, we already have that now. You don't have to have a neural interface embedded in your brain to get access to essentially all information in the world — you just have to have your iPhone. Whether you could do it faster through a brain interface, I definitely wouldn't rule that out. But think about this: the systems that we have already to speak and to communicate have evolved over thousands and millions of years and are supported by neural structures with bandwidth of millions of neurons. There is no technology that exists right now, certainly not in commercial form and not even in research labs, that comes anywhere close to what has been evolved for those natural purposes.
So I'm essentially saying two sides of this: we're already getting into this now, this is not new territory, the topic of augmentation both physical and cognitive. But we are entering this area of enhanced cognition, and I think the technology is going to be the rate-limiting step in how far we can go. We have not had the full conversations about — number one — is this what we actually want? Is this going to be good for society? Who gets access to this technology? These are all things that are going to become real-world problems.
Avatar Communication and Facial Expression Decoding
Andrew Huberman: Could you tell us what you're doing in terms of merging the brain-machine interface with extraction of speech signals from people who are locked in, like Poncho, with facial expressions?
Dr Eddie Chang: We're here together in person. We could have done this virtually, but there is something about being able to actually see your expressions and understand other forms of communication. Nonverbal expressions are a really important part of the way we speak. If you have a quizzical look on your face when I'm saying something unclear, that's a sign to me that I need to rephrase it, say it in a different way, or slow down. Facial expressions are a really important part of the way we communicate. And it's not just the expressions of how you're feeling and perceiving what I'm saying — it's also seeing my mouth move. Your eyes actually see my mouth move and my jaw move in a particular way that allows you to hear those sounds better. Having both the visual information and the sounds go into your brain improves intelligibility and makes it more natural.
The reason why we're also very interested in this idea of not just having text on a screen, but essentially a fully computer-animated face — an avatar of the person's speech movements and their facial expressions — is that it's going to be a more complete form of expression. Right now that might just be someone looking at a computer screen interpreting these signals. But I think the way things are going in the next couple of years, a lot more of our social interactions are going to move into this digital virtual space. Most people are thinking about what that means for most consumers, but it also has really important implications for people who are disabled — how are they going to participate in that? So we were thinking about for people like Poncho and other people who are paralysed, what other forms of brain-machine interface can we do in order to help improve their ability to communicate. One is essentially building out more holistic avatars — things that can decode essentially their expressions or the movements associated with their mouth and jaw when they actually speak, to improve that communication.
Andrew Huberman: Do you envision a time not too long from now where instead of tweeting out something in text, my avatar will — I'll type it out, but my avatar will just say it? It'll be an image of my avatar saying whatever it is I happen to be tweeting at that moment.
Dr Eddie Chang: That's what we're working on. That is going to happen and it's going to happen soon, and there's a lot of progress in that. We're just trying to enrich the field of communication and expression to make it more normal. We actually think that having that kind of avatar is a way of getting feedback to people learning how to speak through a speech neuroprosthetic — that's the device we call it, a speech neuroprosthetic. That is going to be the way that can help people learn how to do it the quickest. Not necessarily trying to say words and having them come on a screen, but actually having people embody it, feel like it's part of themselves, or that they are directly controlling that illustration or animation.
The Neuroscience of Stuttering
Andrew Huberman: I get a lot of questions about stuttering. What can people with a stutter do if they'd like to relieve it?
Dr Eddie Chang: Stuttering is a condition where the words can't come out fluently. You have all the ideas, you've got the language — remember we talked about this distinction between language and speech. Stuttering is a problem of speech. The ideas, the meanings, the grammar — it's all there in people who stutter, but they can't get the words out fluently. So that's a speech condition, and in particular it's a condition that affects articulation — specifically about controlling the production of words in this really coordinated way that has to happen in the vocal tract to produce fluent speech.
Stuttering is a condition where people have a predisposition to it. You are a stutterer or you're not a stutterer. But people who stutter don't stutter all the time. You could be a stutterer who stutters sometimes but not others. The main link between stuttering and anxiety is that anxiety can provoke it and make it worse. That's certainly true, but it's not necessarily caused by anxiety. It can trigger it or make it worse, but it's not the cause of it per se. The cause of it is still really not clear, but it does have to do with these brain functions that we've been talking about earlier — in order to produce normal fluent speech, we're not even conscious of what is going on in our mouths, in our larynx. If we were, we would not be able to speak, because it's too complex, too precise. It's something that we've developed the abilities to do and we do it naturally. It's part of our programming, part of what we learn inherently through exposure.
Stuttering is essentially a breakdown at certain times in that machinery being able to work in a really coordinated way. You can think about the operations of these areas that are controlling the vocal tract — speech is like a symphony. In order for it to come out normally, you've got to have not just one part, the larynx, but the lips, the jaw — they can't be doing their own thing. They have to be very, very precisely activated and controlled in a way to actually create words. In stuttering, there's a breakdown of that coordination.
Andrew Huberman: If somebody has a stutter, is it better to address that early in life when neuroplasticity is very robust? And if so, what's the typical route for treatment? I have to imagine it's not brain surgery typically. I'm guessing there are speech therapists that people can talk to, and they can help them work out where they're getting stuck in relation to anxiety.
Dr Eddie Chang: Yeah, exactly. Part of it is about that anxiety, but a lot of it really has to do with therapy to work through and think of tricks, basically, to create conditions where you can actually get the words to come out. A lot of some forms of stuttering are really initiation problems — just getting started itself is very hard. You want to start with an initial vowel or consonant, but it won't emit. So a lot of the therapy is really just focusing on how do you create the conditions for that to happen.
There's another aspect to it that I find very interesting, which is auditory feedback — essentially what we hear ourselves say. Every time I say a word, I'm also hearing what I'm saying. That turns out to be very important. Sometimes when you change that, it can actually change the amount someone stutters, for better or for worse. It's giving us a clue that the brain is not just focused on sending the commands out, but it's also possibly interacting with the part that is hearing the sounds. There's something that might be going on in that connection that breaks down when stuttering occurs.
So there are individuals who are stutterers but don't stutter all the time. In those instances, there's something happening in those particular moments where this very, very precise coordination needs to happen in the brain in order to get the words out fluently.
Closing Remarks
Andrew Huberman: Eddie, I have to say, from the first time we became friends — 38 years ago —
Dr Eddie Chang: Something like that.
Andrew Huberman: Something like that. To be sitting here with you today is an absolute thrill for me. Not just because we've been friends for that long, or that we got reacquainted through literally the halls of medicine and science, but because I really do see what you're doing as representing the absolute cutting edge of exploration and application. The story of Poncho is but one of your many patients who has tremendously benefited from your work. And now as chair of a department, you of course work alongside individuals who are also doing incredible work in the spinal cord and beyond. So on behalf of myself and everyone listening, I just really want to thank you for joining us today to share this information, but also just for the work you do. It's truly spectacular. Thank you ever so much.
Dr Eddie Chang: Thanks.