Film and TV Scenes That Make No Sense without Non-Speech Captions
Updated: January 4, 2018
In the film Ferris Bueler’s Day Off, Ferris uses a keyboard to simulate coughing, sneezing, and hacking noises to convince people over the phone that he is sick and cannot come to school.
Watching a movie or TV show with bad captions can be exhausting.
Proper captions are supposed to give d/Deaf and hard of hearing viewers, and those watching without sound, an equivalent experience. That means that non-speech sounds and other aural cues must be included in captions because they are crucial to recreating the intended “audience experience” without using audio. And when those non-speech elements are missing, viewers end up confused, annoyed, and eventually disinterested with that video content.
Automated speech recognition (ASR) technology, although sometimes impressively accurate, comes up short when it comes to identifying key non-speech elements of captioning. Non-speech elements include sound effects, speaker labels that indicate who is talking, and other sonic information that might be necessary to properly follow the plot or dialogue.
1. “No Audio” Scenes in The Life Aquatic
Sometimes, it is necessary to indicate to the viewer that they aren’t missing anything at all.
There are several scenes in Wes Anderson’s loveably whimsical Bill Murray vehicle The Life Aquatic in which characters are seen having conversations but no speech is actually heard. Music is playing over the conversations and that is the only audio the viewer is meant to hear.
If it were not for the captions that read “(no audio)” each time the camera cuts to a conversation in these situations, the viewers would assume something was left out of the captions when they were just supposed to hear the music.
2. Different Uses of ASL in Master of None and Fargo
ASL in Master of None with subtitles. Image: Netflix
ASL in Fargo without subtitles. Image: Hulu
Not every language relies on sound to facilitate communication. For example, sign language — something which ASR cannot currently recognize and render as captions on screen.
On a memorably standalone episode of Netflix’s Master of None, (American sign language) ASL is the primary language used on screen. In this case, it’s necessary to include non-speech captions otherwise people who do not understand ASL would be excluded from the plot. Technically, non-speech captions for ASL are actually called subtitles because they translate one language to another, but this example still deserves a place on this list nonetheless.
In FX’s original series Fargo, however, the captions — or subtitles — for ASL were left out intentionally. But why?
This is one of those cases where the shows’ producers decided they want to leave the audience in the dark as to what was being signed on screen. In the first season of Fargo, we are introduced to two imposing-looking mob members who both use ASL to communicate (only one speaks English). Assuming the viewer does not know ASL, the absence of subtitles for sign language is strategic because it keeps the viewer in suspense during certain scenes, which compliments the general tone of the show very well.
3. “Upsidedown World” Sounds in Stranger Things
In this scene from Netflix’s Stranger Things, two characters named Mike and Eleven are having a conversation over the subtle whirring, crackling, and whining noises of a staticky radio.
The radio sounds are integral to the dialogue in this scene. Mike shouts to Eleven (who is toggling with the settings on the radio) a couple of times to turn it off when it is making noises off-screen. Their conversation makes little sense without the context of the radio’s annoying, staticky interruptions.
Later, to the surprise of the viewer (without giving too much away…) a certain someone’s voice comes over the radio from the “Upsidedown world.” Without the foreshadowing of the radio sound-effect captions at the beginning of the scene, this new voice is doubly surprising and probably a little confusing for someone trying to follow the scene at the intended pace.
4. Setting Up a Joke in The IT Crowd
It’s always a shame when a good joke goes unnoticed because computer-generated captions couldn’t set up for the punch line.
The UK’s Channel 4 witty sitcom The IT Crowd offers up a great example of proper captioning for comical situations.
In one opening scene of the first season, jaded slacker and corporate technical support worker Roy is reclining at his desk reading a magazine. The captions read “(Singing) We don’t need no education.” To most people listening to the audio, this is immediately recognized as a part of a song you’ve heard before (Pink Floyd’s classic “Another Brick in the Wall”). Without the caption (Singing), however, you would not know he was singing. You might just assume he was stating that we, in fact, do not need no education.
That (Singing) caption is important becuase his nerdy co-worker Moss responds with, “Yes you do; you’ve just used a double negative.” That punchline isn’t really a punchline without that non-speech caption. Without that context, it’s just the next line in what appears to be a conversation.
5. The Whisper: Lost In Translation
Image: Amazon Video
Okay, one more Bill Murray movie before we go…
Sofia Coppola’s Oscar-winning triumph Lost in Translation has, arguably, one of the most memorable final scenes in recent cinema.
At the very end of the film, Murray’s character Bob famously whispers some brief, parting words into the ear of Charlotte, played by Scarlett Johansson.
What did he say? No one really knows. The whisper is intended to be inaudible. If you had the sound off or couldn’t hear, however, you would have no idea. You would have thought you missed something critical to the movie’s conclusion.
The intention of the inaudible whisper is probably to allow the viewer to use their imagination as to what Murray’s character said. There is, however, plenty of debate about what Murray actually said to Johansson when that scene was filmed if you want to try and decide for yourself!
Captions Make Art Accessible, Not Just Video
Coppola’s decision to end Lost in Translation with ambiguity is part of what makes this movie such a classic and distinguishes her as a director. This is why proper captioning is so important; so that all viewers can appreciate why certain moments in film and TV are so exceptional and worthy of our attention or praise.
In entertainment, proper captioning standards should be taken as seriously as the filmmaking process itself if the intention is for everyone to enjoy the content equally — with or without sound. Because everyone deserves to be a critic, including those who cannot hear.
To learn more about captioning for non-speech sounds, visit ReadingSounds.net.
5 Publishing Firms Doing Captioning Right
In the world of publishing, people are going digital. As a result, this outburst of digital content has created greater access to educational materials for a wider range of people. While digital content is easier to disseminate, it can also be made…
Q&A: McGraw-Hill’s Roadmap Towards Greater Accessibility
Through their Roadmap to Accessibility, McGraw-Hill is steadily incorporating its accessibility initiatives into their products. As a result, McGraw-Hill is becoming a leader in accessible publishing. While they are the first to admit that it’s not always a clear road ahead, McGraw-Hill’s…
4 Reasons You Need Caption Encoding
What is it? Caption encoding is when captions are embedded into the video and presented as a single asset. Typically, captions are added onto a video as a “sidecar file,” but this method is intended for online video where one can upload…