In the film Ferris Bueler’s Day Off, Ferris uses a keyboard to simulate coughing, sneezing, and hacking noises to convince people over the phone that he is sick and cannot come to school.
Watching a movie or TV show with bad captions can be exhausting.
Proper captions are supposed to give d/Deaf and hard of hearing viewers, and those watching without sound, an equivalent experience. That means that non-speech sounds and other aural cues must be included in captions because they are crucial to recreating the intended “audience experience” without using audio. And when those non-speech elements are missing, viewers end up confused, annoyed, and eventually disinterested with that video content.
Automated speech recognition (ASR) technology, although sometimes impressively accurate, comes up short when it comes to identifying key non-speech elements of captioning. Non-speech elements include sound effects, speaker labels that indicate who is talking, and other sonic information that might be necessary to properly follow the plot or dialogue.
White Paper: Closed Captioning Best Practices for Media & Entertainment
To demonstrate why this aspect of captioning is so important, here is a list of examples from an assortment of TV shows and films where non-speech captions are an absolute necessity.
1. “No Audio” Scenes in The Life Aquatic
Sometimes, it is necessary to indicate to the viewer that they aren’t missing anything at all.
There are several scenes in Wes Anderson’s loveably whimsical Bill Murray vehicle The Life Aquatic in which characters are seen having conversations but no speech is actually heard. Music is playing over the conversations and that is the only audio the viewer is meant to hear.
If it were not for the captions that read “(no audio)” each time the camera cuts to a conversation in these situations, the viewers would assume something was left out of the captions when they were just supposed to hear the music.
2. Different Uses of ASL in Master of None and Fargo
ASL in Master of None with subtitles. Image: Netflix
ASL in Fargo without subtitles. Image: Hulu
Not every language relies on sound to facilitate communication. For example, sign language — something which ASR cannot currently recognize and render as captions on screen.
On a memorably standalone episode of Netflix’s Master of None, (American sign language) ASL is the primary language used on screen. In this case, it’s necessary to include non-speech captions otherwise people who do not understand ASL would be excluded from the plot. Technically, non-speech captions for ASL are actually called subtitles because they translate one language to another, but this example still deserves a place on this list nonetheless.
In FX’s original series Fargo, however, the captions — or subtitles — for ASL were left out intentionally. But why?
This is one of those cases where the shows’ producers decided they want to leave the audience in the dark as to what was being signed on screen. In the first season of Fargo, we are introduced to two imposing-looking mob members who both use ASL to communicate (only one speaks English). Assuming the viewer does not know ASL, the absence of subtitles for sign language is strategic because it keeps the viewer in suspense during certain scenes, which compliments the general tone of the show very well.
3. “Upsidedown World” Sounds in Stranger Things
In this scene from Netflix’s Stranger Things, two characters named Mike and Eleven are having a conversation over the subtle whirring, crackling, and whining noises of a staticky radio.
The radio sounds are integral to the dialogue in this scene. Mike shouts to Eleven (who is toggling with the settings on the radio) a couple of times to turn it off when it is making noises off-screen. Their conversation makes little sense without the context of the radio’s annoying, staticky interruptions.
Later, to the surprise of the viewer (without giving too much away…) a certain someone’s voice comes over the radio from the “Upsidedown world.” Without the foreshadowing of the radio sound-effect captions at the beginning of the scene, this new voice is doubly surprising and probably a little confusing for someone trying to follow the scene at the intended pace.
4. Setting Up a Joke in The IT Crowd
It’s always a shame when a good joke goes unnoticed because computer-generated captions couldn’t set up for the punch line.
The UK’s Channel 4 witty sitcom The IT Crowd offers up a great example of proper captioning for comical situations.
In one opening scene of the first season, jaded slacker and corporate technical support worker Roy is reclining at his desk reading a magazine. The captions read “(Singing) We don’t need no education.” To most people listening to the audio, this is immediately recognized as a part of a song you’ve heard before (Pink Floyd’s classic “Another Brick in the Wall”). Without the caption (Singing), however, you would not know he was singing. You might just assume he was stating that we, in fact, do not need no education.
That (Singing) caption is important becuase his nerdy co-worker Moss responds with, “Yes you do; you’ve just used a double negative.” That punchline isn’t really a punchline without that non-speech caption. Without that context, it’s just the next line in what appears to be a conversation.
5. The Whisper: Lost In Translation
Image: Amazon Video
Okay, one more Bill Murray movie before we go…
Sofia Coppola’s Oscar-winning triumph Lost in Translation has, arguably, one of the most memorable final scenes in recent cinema.
At the very end of the film, Murray’s character Bob famously whispers some brief, parting words into the ear of Charlotte, played by Scarlett Johansson.
What did he say? No one really knows. The whisper is intended to be inaudible. If you had the sound off or couldn’t hear, however, you would have no idea. You would have thought you missed something critical to the movie’s conclusion.
The intention of the inaudible whisper is probably to allow the viewer to use their imagination as to what Murray’s character said. There is, however, plenty of debate about what Murray actually said to Johansson when that scene was filmed if you want to try and decide for yourself!
Captions Make Art Accessible, Not Just Video
Coppola’s decision to end Lost in Translation with ambiguity is part of what makes this movie such a classic and distinguishes her as a director. This is why proper captioning is so important; so that all viewers can appreciate why certain moments in film and TV are so exceptional and worthy of our attention or praise.
In entertainment, proper captioning standards should be taken as seriously as the filmmaking process itself if the intention is for everyone to enjoy the content equally — with or without sound. Because everyone deserves to be a critic, including those who cannot hear.
To learn more about captioning for non-speech sounds, visit ReadingSounds.net.
Closed captions help students study more efficiently, watch videos in sound-sensitive environments, and are an accommodation for those who are deaf or hard of hearing…. Read More
Accessibility lawsuits can tell us a lot about what makes an organization accessible to both employees and customers. Two lawsuits in particular, against international shipper… Read More
Amazon is no stranger to the need for – and the benefits of – accessible products. Many of Amazon’s employees have disabilities themselves. These individuals… Read More
Video has transformed the way institutions of higher education approach teaching. As a flexible, customizable, and creative platform, video has paved the way for greater… Read More
It’s not often that people ask for stricter enforcement of laws. When it comes to web accessibility in the UK, that’s exactly what’s happening. AbilityNet… Read More
We have some exciting webinars coming to you that you won’t want to miss. This fall, we’ll be talking all things legal, from the Winn-Dixie… Read More
It’s about to get cloudy up in here because… We are excited to announce you can now upload videos for captioning and audio description via… Read More
What Is Dyslexia? Dyslexia is a general term used to describe difficulties with learning to read and interpret letters, symbols, and words. It is one… Read More
In order to survive in modern day journalism, news outlets must keep up with the trends. And the biggest trend of the century has undeniably… Read More