Seeing is believing

eye rf

If you’ve ever done pronunciation drilling with your hand cupped behind your ear like a demented kids TV presenter (all together now!) you’re probably of the opinion that exaggerating audio cues helps students to discriminate the sounds of a second language. But did you know that students can improve their chances of learning sounds by simply looking at a speaker’s face?

Research on how language learners use visual cues in the perception and production of sounds supports what auditory and visual sciences have known for some time: what you see has a strong impact on what you hear. This relationship is niftily illustrated through the “McGurk effect”, not the latest McDonald’s menu, but a rather compelling example of how the human sensory experience is interlinked.

More recently, with the help of fMRI scans (which basically involve sending people down a tube into a giant washing machine and having a nosy around at changes in cerebral blood flow) neuroscientists have found that the auditory cortex is active while watching silent speech (Calvert et al. 1997). In other words, the part of the brain that usually deals with sound is affected by visual information, which helps native English speakers to identify sounds in face-to-face communication. The good news is, it seems that visual cues can be interpreted by non-native speakers too.

Studies suggest that audiovisual training, in which students are shown a video of the speaker’s face, is more effective than listening practice alone in helping students to differentiate sounds (e.g. Hazan et al. 2005). Unsurprisingly, this technique has been shown to be most effective in sounds where the difference in mouth position is highly visible, such as b and v (a saving grace for Spanish students and their English bowel problems). However, audiovisual training has also produced positive results for problem areas with more subtle physical differences, for example in the differentiation of l and r for Japanese students. What’s more, these studies show a positive impact on pronunciation, indicating that increased attention to native speakers’ mouth movements enables students to reproduce the sounds more effectively themselves.

Most teachers agree that out of the big four (speaking, listening, reading and writing), it’s listening which causes students a real pain in the auricle. Given that the majority of real world listening takes place face-to-face, excluding visual components a priori is at odds with what we know about natural speech processing. In light of studies which point to the advantages of visual training, it’s time to question the dominance of audio-only files in the classroom.

  • Do you do any exercises which draw students’ attention to the speaker’s mouth?
  • How do you think this research could be applied in classrooms which don’t have video technology?
  • Can you think of any practical ways to help students to pay attention to visual cues?

Calvert, G., Bullmore, E., Brammer, M., Campbell, R., Williams, S., McGuire, P., Woodruff, P., Iversen, S and David, A. (1997) Activation of Auditory Cortex During Silent Lipreading. Science 25: 276

Hazan, V., Sennema, A., Midori, I., Faulkner, A. (2005) Effect of audiovisual perceptual training on the perception and production of consonants by Japanese learners of English. Speech Communication 47: 360



Filed under Articles

14 responses to “Seeing is believing

  1. Reblogged this on adventures in teaching and commented:
    I love this one, I’ll use it in class!

  2. Thank you for this article and “Linguistics of Babies”. For years I have struggled with pronouncing certain Dutch words. I thought with my German background it would be easy to learn the language but my son who lives in the Netherlands is constantly correcting my pronunciation. To be honest, most of the time I don’t hear the difference. It’s very frustrating.

  3. Thanks for the follow, and for leading me to your blog. I find your studies fascinating! I am not a teacher or a scholar, just someone who is learning a second language as an adult. I have always loved sounds and languages, but never had the time or resources to try to learn until now. I am learning French for an upcoming vacation. I did this solely with internet resources and connections for the first year and several months. I found a native French teacher last summer and have made leaps and bounds in my conversation skills thanks to her. I think it’s very true about the visual aspect, which she does emphasize. I also use Skype with native speakers. Thanks to technology, all this is now possible! Anyway, I look forward to reading more of your work. Thanks!

  4. Pingback: I didn’t watch! | Forty, c'est Fantastique !

  5. Reblogged this on A Work in Digress. and commented:
    Mind blown. This is a truly fascinating phenomenon that I’m sure must have some kind of bearing on learning languages. Having moved to Germany a month ago, I feel my level of German has certainly improved since moving here, and other than being immersed in the language, I think human interaction (watching faces and listening before then speaking) is the principle reason for the speed with which it has.

  6. When I taught articulatory phonetics in Costa Rica, I found it very satisfying to be able to show people objectively (with sketches on the blackboard or from the textbook) exactly how sounds are made. I have no idea how much phonetics training ESL teachers receive, but I think it would be extremely helpful.

    It was particularly fun to teach aspirated stops and English diphthongs and then set the students to pronouncing Spanish phrases with gringo pronunciation: ‘tres mil’ becomes “trace meal”, etc.

    Your point about watching a speaker’s face is very important. Ventriloquists are able to fool native speakers because there is not that much auditory difference between the various sounds made in the same part of the mouth. So a non-native speaker who already has the challenge of differentiating unfamiliar sounds or contrasts will find working with audio-only recordings doubly difficult. By watching a speaker’s face, the learner can notice subtle cues (lip rounding with /r/, for instance, which we don’t tend to do with /l/).

  7. This is fascinating research – thank you. Friends with one deaf child and one with normal hearing have taught both sign language, and the visual links between gestural and aural communications are fascinating: I’m sure they’re improvising on occasion.

  8. Reblogged this on 'Speak in English' with Confidence and Ease and commented:
    We at Tricks- Language Lab use a lot of visual clues and blended learning with multimedia is part of our methodology… this is an interesting article on using the face of the speaker.

    To answer to your Q3: I as them to do learning style assessment 24 questions on how they learn the best. This helps me understand who are more of visual learners than the rest of the groups… and if the group loves it i like to use a lot of media. Thanks to BBC’s 6-min. videos and some licensed content that I have picked-up…

  9. Fascinating! I have learned French and Spanish in college but without using those languages have lost a lot of ability. Still, when travelling I find words come back to me when conversing person to person. Watching a face is key!

  10. Excellent! I know I hear better when I can watch someone’s mouth.
    My husband speaks to me facing the other way and I can’t hear a thing!
    ( I do have hearing aids!)

  11. kidgriff

    Totally true, I’ve lived in Spain for nearly 6 years but still have trouble understanding people over the phone. But when face to face, I understand perfectly.

  12. As an English teacher to young Spanish speaking children I am constantly telling them, “Mirame.” (“Look at me.”) They don’t hear the difference, but they can see that my teeth are touching my lips when I say “v” and my lips are touching together when I say “b.” When they’re struggling, I describe what’s happening in my mouth – my lips are touching, my lips aren’t touching, my tongue is here or my tongue is there. They love to try and are so excited when I tell them, “Good job!”

  13. Pingback: Seeing is believing | From a teacher's point of view, some is old and some is new!

  14. Case in point why trying to understand a non-native language over the phone is fiendishly difficult!

    A German woman I know got a telemarketing job soon after she moved to the U.S. Not only did she have to deal with a lack of facial cues, but she was faced with the variety of regional accents. I don’t know how she managed!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s