Neuroscience
Forget lip-reading, the way the top of the head moves as we speak also plays a part in communication, a finding that has implications for creating more realistic animated characters.
In an initial experiment, Chris Davis and Jeesun Kim presented students with silent videos that showed the top half of a man’s head as he read out various sentences.
When presented with two pairs of such videos, one in which the man read the same sentence out in each video, and another in which he read different sentences, the students were able to use the movement of the top of his head to judge better than chance which pair was the same and which was different. Note that in the matching pair, the videos were not identical – the man was recorded reading the same sentence on two separate occasions.
Performance was also better than chance when the students had to match the sound of the man reading a sentence with the correct silent video that showed only the top half of his head as he read the same sentence.
In both cases, performance was better when the man was reading an expressive sentence like “that is really annoying; I have to let you know” rather than a mundane sentence like “the jacket hung on the back of the wide chair”.
In a further experiment, the sound of a person speaking was deliberately distorted. This time, the student participants were able to correctly recognise more words if the sound was accompanied by a video showing only the outline of the top of half of the head of the person who was speaking. Again, the sight of the top of the head provided greater advantage if the speaker was uttering an expressive sentence.
“That people are sensitive to speech related upper head movements makes it clear that the production of natural looking virtual characters will need to consider more than the correct animation of the mouth and jaw”, the researchers said.
__________________________________
Davis, C. & Kim, J. (2006). Audio-visual speech perception off the top of the head. Cognition, 100, B21-B31.
Post written by Christian Jarrett (@psych_writer) for the BPS Research Digest.
- Second Language Changes The Way Bilinguals Read In Their Native Tongue
Do bilinguals have an internal switch that stops their two languages from interfering with each other, or are both languages always "on"? The fact that bilinguals aren't forever spurting out words from the wrong language implies there's some kind...
- The Effect Of, Er, Hesitations In Speech
We'd all like to speak with unobstructed lucidity, but for most of us speech hesitations and pauses are unavoidable. Now psychologists have asked what effect these oral hiatuses have on listeners. Martin Corley and colleagues played recordings of...
- Reading Is A Team Effort
Psychologists Denis Pelli and Katharine Tillman have shown that reading is a team effort in the sense that the three reading processes of letter decoding, whole world recognition, and using sentence context, each make a unique, additive contribution to...
- Meaningful Gestures Help People Remember What You Said
Complement your speech with gestures if you want people to remember what you’re telling them, but make sure they’re related to what you’re saying and not just random gesticulations. Pierre Feyereisen at the University of Louvain in Belgium showed...
- Mimicry The Best Form Of Flattery For Computers Too
After a social interaction, participants rate people who mimicked their movements or speech more favourably than they rate people who didn’t mimic them, even if they were unaware of the mimicry – a phenomenon that’s been dubbed the chameleon effect....
Neuroscience
Communicating off the top of your head
Forget lip-reading, the way the top of the head moves as we speak also plays a part in communication, a finding that has implications for creating more realistic animated characters.
In an initial experiment, Chris Davis and Jeesun Kim presented students with silent videos that showed the top half of a man’s head as he read out various sentences.
When presented with two pairs of such videos, one in which the man read the same sentence out in each video, and another in which he read different sentences, the students were able to use the movement of the top of his head to judge better than chance which pair was the same and which was different. Note that in the matching pair, the videos were not identical – the man was recorded reading the same sentence on two separate occasions.
Performance was also better than chance when the students had to match the sound of the man reading a sentence with the correct silent video that showed only the top half of his head as he read the same sentence.
In both cases, performance was better when the man was reading an expressive sentence like “that is really annoying; I have to let you know” rather than a mundane sentence like “the jacket hung on the back of the wide chair”.
In a further experiment, the sound of a person speaking was deliberately distorted. This time, the student participants were able to correctly recognise more words if the sound was accompanied by a video showing only the outline of the top of half of the head of the person who was speaking. Again, the sight of the top of the head provided greater advantage if the speaker was uttering an expressive sentence.
“That people are sensitive to speech related upper head movements makes it clear that the production of natural looking virtual characters will need to consider more than the correct animation of the mouth and jaw”, the researchers said.
__________________________________
Davis, C. & Kim, J. (2006). Audio-visual speech perception off the top of the head. Cognition, 100, B21-B31.
Post written by Christian Jarrett (@psych_writer) for the BPS Research Digest.
- Second Language Changes The Way Bilinguals Read In Their Native Tongue
Do bilinguals have an internal switch that stops their two languages from interfering with each other, or are both languages always "on"? The fact that bilinguals aren't forever spurting out words from the wrong language implies there's some kind...
- The Effect Of, Er, Hesitations In Speech
We'd all like to speak with unobstructed lucidity, but for most of us speech hesitations and pauses are unavoidable. Now psychologists have asked what effect these oral hiatuses have on listeners. Martin Corley and colleagues played recordings of...
- Reading Is A Team Effort
Psychologists Denis Pelli and Katharine Tillman have shown that reading is a team effort in the sense that the three reading processes of letter decoding, whole world recognition, and using sentence context, each make a unique, additive contribution to...
- Meaningful Gestures Help People Remember What You Said
Complement your speech with gestures if you want people to remember what you’re telling them, but make sure they’re related to what you’re saying and not just random gesticulations. Pierre Feyereisen at the University of Louvain in Belgium showed...
- Mimicry The Best Form Of Flattery For Computers Too
After a social interaction, participants rate people who mimicked their movements or speech more favourably than they rate people who didn’t mimic them, even if they were unaware of the mimicry – a phenomenon that’s been dubbed the chameleon effect....