Rethinking How We Use Closed Captioning in Education
Updated: June 3, 2019
Video captions are more than just text on a screen.
That’s what Sean Zdenek believes. The Associate Professor of English at Tech Tech literally wrote the book on captions: Reading Sounds: Closed-Captioned Media and Popular Culture.
Through extensive research, observation, and analysis, Sean probes into how captions work, what they are capable of, and how we can use them to improve communication, entertainment, and education in multimedia.
This last piece was the focus of Sean’s presentation, The Future of Closed Captioning in Higher Education.
Watch a recording of this presentation below, or read on for key take-aways that will have you rethink how we can use closed captioning in education.
A Rhetorical Approach to Captioning
To challenge our assumptions about what closed captioning is, Sean outlines five observations about captioning that reveal their complexity and nuance.
1. Closed captioning is more than just transcription
If closed caption were just rote copying, you could put a team of untrained people to the task.
But it’s much more than that.
2. Captioning is both a skill and an art
Captioning is a skill that at times requires some creative solutions, especially when you’re dealing with non-speech sounds.
3. Captions don’t caption sounds per se
Captions don’t caption sounds; they capture meaning.
Often video includes audio of actions that aren’t on screen. Sean gives a great example of this with a screencap of a scene from The Young Doctor’s Notebook.
The sound indicates that the bartender is turning off the tap. Since we can’t see that happen, the caption must communicate that action through text. In this way “TURNS OFF TAP” is far more meaningful than a caption that says “SQUEAK.”
4. Captions produce new text
When closed captions include speaker IDs with dialogue, the captioner is adding new text into the viewing experience. Sean noted an interesting example of this in the 2013 Superman movie, Man of Steel.
So the movie itself is really kind of playful and coy about that name. It’s rarely uttered. It’s only uttered at the end of the movie. At one point, Lois can’t say the whole name. She can only say “Super” before she’s interrupted.
But on the caption track, Superman appears all over the place as a speaker ID. And for me, this is one indication of how captions can start to produce a different experience of the text. The experience through captions is different in this way from the experience through sound alone.
Another interesting way that captions add text to the viewing experience is how they communicate our assumption about sound production itself. For example, if lips are moving, like mouthed speech, but no sound is coming out, the captioner may have to include a caption “SILENCE,” there or just put “MOUTHED.”
5. Captions transform meaning
Reading captions instead of listening to audio can transform the viewing experience in subtle ways. Sean points out ways that captions offer clarity and context to actions on screen by identifying details we take for granted.
Some great examples of this from Sean’s website:
Music playing in the background is clarified with lyrics and music symbols spelled out in the captions:
When a movie introduces a made-up new word, the caption clarifies it. You can even learn the correct spelling.
Multiple sounds may occur at once, but they can’t be read simultaneously. Captions linearize sound by presenting the soundscape in a form that can be read one sound at a time.
A Practical Approach to Captioning
Sean introduces some fascinating theories about captioning and how to think about it differently. He also covers the more practical aspect of how to do captioning well and why it’s important, especially in education.
He notes that accuracy is paramount, not just for ensuring equal access to information, but also because inaccurate captions get a lot of negative attention (see: caption fails).
It’s also important for the end user to be able to customize the display settings for their video captions, adjusting size, font, color, and background for readability. This is part of the larger goal of honoring individual preferences and building robust systems to support them.
YouTube and Hulu (pictured below) offer flexible options for customizing caption display. Giving options for personalizing display is part of universal design, ensuring that your technology works for everyone.
An Innovative Approach to Captioning
How might we use captions in unexpected ways?
Sean offers lots of innovative ideas for how captions can enhance education:
- Make text engaging with an interactive transcript as the video plays
- Search entire video libraries and lecture archives for keywords or topics in caption text
- Offer subtitles in multiple languages (great for language learning classes!)
- Add an alternative easy reading caption track for K-12 or special needs viewers
- Use a video transcript to easily pull and produce video clips
- Use transcripts as study guides
- Have students add their own caption tracks with video annotations
- Use heat maps to see which segments of video are most popular for students to view
- Give professors more data about what students are searching for, how they use transcripts to study, etc.
- Add open captions so text is available on lectures and videos regardless of device it’s viewed on
And it doesn’t stop there.
Watch Sean’s presentation for more on how to rethink captioning for education.
Video Translation in a Hybrid World
After an unprecedented global pandemic that led to relying solely on digital communications, many people are slowly making their way back to in-person settings. Even as physical gatherings become more commonplace, however, we’re seeing many virtual components stick around in the form…
3 Reasons Why You Need Video Transcription
Video transcription is the process of translating your video’s audio into text using automatic speech recognition technology, human transcriptionists, or a combination of the two. Without video transcription, your videos rely solely on audiovisual material to convey information. What motivates a video…
WebVTT Captions: How to Create a Web Video Text Track File
A “Web Video Text Track” file, also known as WebVTT (.vtt), is a popular subtitle and caption file format. WebVTT was created in 2010 by the Web Hypertext Application Technology Working Group (WHATWG) to support text tracks in HTML5. WebVTT was broadly…