Seeing a band live is a much more engaging experience than listening to a recording at home. But it’s not just the atmosphere of a live event, it’s also the singer’s facial expressions and gestures which enhances our emotional experience.
It’s a rare singer whose face remains impassive as they sing. Think of jazz greats like Ella Fitzgerald, blues artists like B. B. King or pop acts like Michael Jackson. As they sing, their faces are conveying the emotion in the music, sometimes it seems, even struggling with the emotions. Part of the beauty of a live performance is seeing singers’ faces as they interact with both the music, the other musicians and the audience. Their facial expressions often seem part of the music itself.
Facial expressions affect perception of music
In what way, then, do facial expressions while singing affect the perception of the music? There is a well-known phenomenon in psychology called the McGurk effect. This demonstrates that what listeners hear is profoundly affected by what they see. This suggests that singers’ facial movements may have large effects on how we perceive music.
In a series of simple experiments, Thompson, Graham and Russo (2005) showed just how important the McGurk effect is when we are looking at singers, compared to when we only have the sound to go on.
Experiment 1: Some participants listened to blues legend B. B. King audio only, while others listened and watched him. Those who had both video and audio channels rated the level of ‘dissonance’ (when the music was negative or discordant) higher at points in which B. B. King winced his eyes, rolled his head back and shook his upper body.
Experiment 2: When trying to judge the pitch changes between notes, participants watching only the video almost did as well as those only listening to the audio. This is pretty impressive.
Experiment 3: Here some participants saw facial expressions relating to pitch changes that matched the audio, while others saw facial expressions that didn’t. When the video didn’t match the audio people were less accurate at judging pitch.
This is like an auditory McGurk effect. Because the video is suggesting the pitch goes down, people’s average ratings of the pitch is lower despite the fact they can hear exactly how much the pitch is going up.
Experiment 4: This experiment used the fact that sung major intervals (series of notes) sound ‘happy’ and minor intervals sound ‘sad’. In some conditions participants heard a ‘happy’ series of notes but saw a seemingly unhappy singer. In other conditions intervals were consistent with facial expressions. Participants rated the emotional component of the intervals more highly if audio and video were congruent.
Experiment 5: This time participants heard real musical performances, some while watching the accompanying video, some without. Here it was found that sometimes the visual channel added to the performance, and sometimes it reduced it.
These experiments are showing a variety of basic ways in which the facial expression of a performer can have a great influence over our perception of music. Facial expressions and body movements can change the pitch we hear, how much emotion we experience and can interact with the music in surprising ways.
That said, it’s clear that not all the visual aspects of musical performance are good. Some body language that performers use may actively reduce our enjoyment of the music.
Culture and music
Thinking more broadly, it’s interesting to reflect on the cultural context in which music is produced. Thompson, Graham and Russo (2005) point out that it wasn’t until the invention of the phonograph that the aural elements of music were split off from the visual.
Modern technological innovations like the iPOD have further reinforced that separation. The coming of pop videos has created another layer of visual meaning to music. This layer, however, often has little to do with the facial expressions and hand gestures of the artist and more to do with the vision of video producers and directors. It’s not often that a music video is a straight film of the band performing the track.
Not just listening, but also watching music
But perhaps we are turning the corner? With the iPOD video singers’ visual performance has the potential to be reunited with the aural experience of music. Will people return to not just listening to music, but also watching it?
Certainly nowadays many musicians make the majority of their money from live performances of their music rather than from sales of their albums. Indeed some of them are even giving their albums away for free.
This may be part of a cultural recognition that music is more than just sound, it is an interaction between singer and audience, a big bit part of which is the artist’s facial expressions and gestures. It’s this interaction on which we are beginning to place a higher value.
ยป Read more from PsyBlog on the ways music influences mood and the personality secrets hidden in your MP3 player.
Reference
Thompson, W.F., Graham, P., & Russo, F.A. (2005). Seeing music performance: Visual influences on perception and experience. Semiotica, 156, 203-227.
