Can You Caption 360° Video, Virtual Reality, and 3D Video?

January 18, 2018 BY SOFIA ENAMORADO
Updated: July 12, 2019

Video is getting real.

First, it started with 3D video.

In 1922, the first 3D film (The Power of Love) hit theaters. Sure, it was in black and white and not as high tech as Avatar or Shrek 3D, but it was revolutionary, and it set a precedent for many eras of captivating 3D films.

Remember 2010 where virtually all the movies seemed to be 3D?

Yet it wasn’t enough. Face it, when you stuck your hand out to try to catch the snitch during Harry Potter and the Order of the Phoenix in 3D… it was just air.

Forest Gump running from a car. The car looks as if it is coming towards us, as in 3D.
Next, a new digital reality was invented: 360° video. Like a panorama, you could now move within a video, either by dragging your mouse or moving your head left and right while wearing a headset.

It was a completely immersive experience and it exploded. Companies used it to showcase products and events, realtors used it to show houses, and adventurers used it to take viewers along on their epic journeys.

Youtube video of a band in a studio playing there instruments. A mouse moves around the image left and right. We see a trumpet player, a saxophone player, a cellist, a piono player and a drummer. The video keeps playing at the mouse moves around.
But it still wasn’t enough. Sensation was missing from the viewing experience.

So (what felt like overnight), virtual reality was born, giving us the freedom to enter into a video and completely immerse ourselves into that world.

Like a scene straight out of Hackers, we, too, began sporting the VR headset.

While it may be some time before cinemas begin incorporating virtual reality into the moviegoer experience, virtual reality isn’t going away.

Beyond gaming, virtual reality is being used to educate people about living with autism and dementia, to teach surgeries, and to teach history in school.

two men wearing virtual reality headsets are punching the air and moving around a living room.
Yet with all these exciting innovations, accessibility seems to have been put on the back burner.

How do you create an immersive experience equal to that of any other user? Where and how should you place captions so that they don’t disrupt the experience and cause the user to fall back into reality?

The challenge with these new technologies is finding a way to make sure the captions are visible and don’t hinder the experience, no matter where the curious wanderer goes.

With “regular” video (and 3D video), viewers sit in front of a screen and the captions are placed within the images. Only the images move – not the viewer.

But with virtual reality and 360° video, it’s a whole ‘nother ball game.

How to add captions to 3D video, 360° video, and virtual reality

sherlock holmes falling from a building. he looks like he is going to fall out of the screen.
3D Video

With 3D video, adding captions is very straightforward.

First, get your video captioned.

Next, you’ll need to download your captions as .srt. Now, if you place SRT captions in a 3D video, the captions won’t appear in 3D, which obviously isn’t ideal.

The solution is to convert SRT captions into a 3D caption format.

There are three popular captioning formats for 3D video:

  • Substation Alpha (.ssa)
  • Advanced Substation Alpha (.ass)
  • VobSub (.idx/.sub pair)

The simplest method for obtaining 3D captions is to convert your SRT caption file to one of the above formats via

Otherwise, you can convert your captions yourself using software like 3D-Subtitler.

a scene from big bang theory. sheldon has a virtual reality headset on. As he reaches out to touch something while wearing the headset, his roommate leonard puts his head in front, so when sheldon touches leonard he freaks out and takes his headset off, startled.
360° video and virtual reality

When it comes to captioning 360° video or virtual reality, there isn’t much research, information, or innovation in the space.

With 360° video, you can easily add text within your video. The caveat is that you have set text that is placed in a set location in the video. So it doesn’t work if someone is talking constantly.

YouTuber LIFE in 360 has an easy video tutorial you can follow. The steps are outlined below.

How to add text to 360° video For the rookie 360° video creator, who may not have access to the lavish technology that makes captions follow the viewer, there is one simple method to embed *at least a couple of words* into your videos. All it requires is that you have Premier and follow these steps shared by YouTuber LIFE in 360. step one "Stich" your 360° video & add to timeline in Premier. Step two Create a new title that will be placed on the video. Step three Place your title on the location you want it to appear. Step four Adjust the title's size and length. Add effects if desired. Step five Export at the exact resolution your camera provides. Step six Use a spherical viewer like Insta360 Studio to view your video in 360°!
Fortunately, BBC’s research and development team is taking the lead on trying to find a sleek solution.

BBC’s team came up with and tested four potential solutions for captioning 360° video. They asked 24 people who use captions to test each solution out and provide feedback.

The results are outlined below.

  • Evenly spaced: Captions were fixed in three locations, evenly spaced by 120° around the video. While deemed as an easier technique to implement, BBC uncovered that users found it hard to view the captions and video at the same time, and felt as if they were missing content.
  • Follow head immediately: Captions were placed as a head-up display, meaning they would always be in front of the viewer, even as they moved around. Surprisingly, this turned out to be the most well-received option. Users felt the captions were easy to locate and read.
  • Follow head with lag: Like a “head-up display,” the captions were placed in front of the viewer, but if the user moved slightly right or left, the captions would remain where they were. The captions would only move if the user made a “larger head movement.” Although BBC thought this would be the most popular solution, users felt the movement of the captions was unpredictable and “reduced their feeling of immersion.”
  • Appear in front, then fixed: Captions appeared where the user first looked, and they remained in that location throughout the experience. But this option was not favored, as participants felt they either missed the captions or missed out on the video when trying to locate the captions.

BBC’s innovative solutions are a step in the right direction. If we want to make these new, exciting technologies more universal, then developers need to seriously consider accessibility.

Through the BBC study, BBC found that there wasn’t a unanimous vote on which solution was the best, highlighting the importance of testing all solutions to find the most optimal captioning method. In addition, while one captioning method may be optimal for one video or program, it may not be optimal for another.

comparing 360 video with virtual reality. 360 video the viewer has control of where they look, turning left or right like a panorama. virtual reality is an interactive 3-dimensional environment where the user has freedom to move anywhere and touch anything.
Furthermore, BBC only tested captioning for 360° video, which is a completely different experience than virtual reality. While these solutions are good ideas to test with virtual reality, developers must consider the impact of virtual reality sickness, as well as the placement of captions in relation to the scene when building captions into their programs.

Ultimately, captioning and accessibility in 360° video and virtual reality need to be incorporated into the design workflow. That way, captions can be incorporated as an enhancement to the immersion.

Even though we haven’t found the solution to virtual reality captioning, we are still experts in good ol’ closed captioning.
Get started with captioning today!

Pig View Pricing CTA

3play media logo in blue

Subscribe to the Blog Digest

Sign up to receive our blog digest and other information on this topic. You can unsubscribe anytime.

By subscribing you agree to our privacy policy.