Plans & Pricing Get Started Login

Rethinking How We Use Closed Captioning in Education

  • tablet with multimedia icons floating off of it

    Video captions are more than just text on a screen.

    That’s what Sean Zdenek believes. The Associate Professor of English at Tech Tech literally wrote the book on captions: Reading Sounds: Closed-Captioned Media and Popular Culture.

    Through extensive research, observation, and analysis, Sean probes into how captions work, what they are capable of, and how we can use them to improve communication, entertainment, and education in multimedia.

    This last piece was the focus of Sean’s presentation, The Future of Closed Captioning in Higher Education.

    Watch a recording of this presentation below, or read on for key take-aways that will have you rethink how we can use closed captioning in education.


    A Rhetorical Approach to Captioning

    To challenge our assumptions about what closed captioning is, Sean outlines five observations about captioning that reveal their complexity and nuance.

    1. Closed captioning is more than just transcription

    If closed caption were just rote copying, you could put a team of untrained people to the task.

    But it’s much more than that.

    2. Captioning is both a skill and an art

    Captioning is a skill that at times requires some creative solutions, especially when you’re dealing with non-speech sounds.

    3. Captions don’t caption sounds per se

    Captions don’t caption sounds; they capture meaning.

    Often video includes audio of actions that aren’t on screen. Sean gives a great example of this with a screencap of a scene from The Young Doctor’s Notebook.

    a bartender with their back turned. Caption reads [TURNS OFF TAP]

    The sound indicates that the bartender is turning off the tap. Since we can’t see that happen, the caption must communicate that action through text. In this way “TURNS OFF TAP” is far more meaningful than a caption that says “SQUEAK.”

    4. Captions produce new text

    When closed captions include speaker IDs with dialogue, the captioner is adding new text into the viewing experience. Sean noted an interesting example of this in the 2013 Superman movie, Man of Steel.

    The word Superman is only uttered three times. But in the captions, the speaker ID for Superman is mentioned, I think, it’s 14 times if you follow that link.

    So the movie itself is really kind of playful and coy about that name. It’s rarely uttered. It’s only uttered at the end of the movie. At one point, Lois can’t say the whole name. She can only say “Super” before she’s interrupted.

    But on the caption track, Superman appears all over the place as a speaker ID. And for me, this is one indication of how captions can start to produce a different experience of the text. The experience through captions is different in this way from the experience through sound alone.

    Another interesting way that captions add text to the viewing experience is how they communicate our assumption about sound production itself. For example, if lips are moving, like mouthed speech, but no sound is coming out, the captioner may have to include a caption “SILENCE,” there or just put “MOUTHED.”

    5. Captions transform meaning

    Reading captions instead of listening to audio can transform the viewing experience in subtle ways. Sean points out ways that captions offer clarity and context to actions on screen by identifying details we take for granted.

    Some great examples of this from Sean’s website:

    Music playing in the background is clarified with lyrics and music symbols spelled out in the captions:

    A man looking disgruntled. Caption reads 'music note – That's why they call me Mr. Fahrenheit – music note'

    When a movie introduces a made-up new word, the caption clarifies it. You can even learn the correct spelling.

    Harry Potter. Caption reads 'No? Well, a Patronus is a kind of positive force.'

    Multiple sounds may occur at once, but they can’t be read simultaneously. Captions linearize sound by presenting the soundscape in a form that can be read one sound at a time.

    An alien monster sniffing the face of a cowering woman. Caption reads: (low hissing, growling) (whimpering, crying)

    A Practical Approach to Captioning

    Sean introduces some fascinating theories about captioning and how to think about it differently. He also covers the more practical aspect of how to do captioning well and why it’s important, especially in education.

    He notes that accuracy is paramount, not just for ensuring equal access to information, but also because inaccurate captions get a lot of negative attention (see: caption fails).

    It’s also important for the end user to be able to customize the display settings for their video captions, adjusting size, font, color, and background for readability. This is part of the larger goal of honoring individual preferences and building robust systems to support them.

    YouTube and Hulu (pictured below) offer flexible options for customizing caption display. Giving options for personalizing display is part of universal design, ensuring that your technology works for everyone.

    Screenshot of The Simpsons playing in Hulu with various color, font, and size options displayed

    An Innovative Approach to Captioning

    How might we use captions in unexpected ways?

    Sean offers lots of innovative ideas for how captions can enhance education:

    • Make text engaging with an interactive transcript as the video plays
    • Search entire video libraries and lecture archives for keywords or topics in caption text
    • Offer subtitles in multiple languages (great for language learning classes!)
    • Add an alternative easy reading caption track for K-12 or special needs viewers
    • Use a video transcript to easily pull and produce video clips
    • Use transcripts as study guides
    • Have students add their own caption tracks with video annotations
    • Use heat maps to see which segments of video are most popular for students to view
    • Give professors more data about what students are searching for, how they use transcripts to study, etc.
    • Add open captions so text is available on lectures and videos regardless of device it’s viewed on

    And it doesn’t stop there.

    Watch Sean’s presentation for more on how to rethink captioning for education.

Leave a Reply

Your email address will not be published. Required fields are marked *

Interested in Learning More?