Can You Caption 360° Video, Virtual Reality, and 3D Video?
Updated: July 12, 2019
Video is getting real.
First, it started with 3D video.
In 1922, the first 3D film (The Power of Love) hit theaters. Sure, it was in black and white and not as high tech as Avatar or Shrek 3D, but it was revolutionary, and it set a precedent for many eras of captivating 3D films.
Remember 2010 where virtually all the movies seemed to be 3D?
Yet it wasn’t enough. Face it, when you stuck your hand out to try to catch the snitch during Harry Potter and the Order of the Phoenix in 3D… it was just air.
It was a completely immersive experience and it exploded. Companies used it to showcase products and events, realtors used it to show houses, and adventurers used it to take viewers along on their epic journeys.
So (what felt like overnight), virtual reality was born, giving us the freedom to enter into a video and completely immerse ourselves into that world.
Like a scene straight out of Hackers, we, too, began sporting the VR headset.
While it may be some time before cinemas begin incorporating virtual reality into the moviegoer experience, virtual reality isn’t going away.
Beyond gaming, virtual reality is being used to educate people about living with autism and dementia, to teach surgeries, and to teach history in school.
How do you create an immersive experience equal to that of any other user? Where and how should you place captions so that they don’t disrupt the experience and cause the user to fall back into reality?
The challenge with these new technologies is finding a way to make sure the captions are visible and don’t hinder the experience, no matter where the curious wanderer goes.
With “regular” video (and 3D video), viewers sit in front of a screen and the captions are placed within the images. Only the images move – not the viewer.
But with virtual reality and 360° video, it’s a whole ‘nother ball game.
How to add captions to 3D video, 360° video, and virtual reality
With 3D video, adding captions is very straightforward.
First, get your video captioned.
Next, you’ll need to download your captions as .srt. Now, if you place SRT captions in a 3D video, the captions won’t appear in 3D, which obviously isn’t ideal.
The solution is to convert SRT captions into a 3D caption format.
There are three popular captioning formats for 3D video:
- Substation Alpha (.ssa)
- Advanced Substation Alpha (.ass)
- VobSub (.idx/.sub pair)
The simplest method for obtaining 3D captions is to convert your SRT caption file to one of the above formats via subtitle3d.com.
Otherwise, you can convert your captions yourself using software like 3D-Subtitler.
When it comes to captioning 360° video or virtual reality, there isn’t much research, information, or innovation in the space.
With 360° video, you can easily add text within your video. The caveat is that you have set text that is placed in a set location in the video. So it doesn’t work if someone is talking constantly.
YouTuber LIFE in 360 has an easy video tutorial you can follow. The steps are outlined below.
BBC’s team came up with and tested four potential solutions for captioning 360° video. They asked 24 people who use captions to test each solution out and provide feedback.
The results are outlined below.
- Evenly spaced: Captions were fixed in three locations, evenly spaced by 120° around the video. While deemed as an easier technique to implement, BBC uncovered that users found it hard to view the captions and video at the same time, and felt as if they were missing content.
- Follow head immediately: Captions were placed as a head-up display, meaning they would always be in front of the viewer, even as they moved around. Surprisingly, this turned out to be the most well-received option. Users felt the captions were easy to locate and read.
- Follow head with lag: Like a “head-up display,” the captions were placed in front of the viewer, but if the user moved slightly right or left, the captions would remain where they were. The captions would only move if the user made a “larger head movement.” Although BBC thought this would be the most popular solution, users felt the movement of the captions was unpredictable and “reduced their feeling of immersion.”
- Appear in front, then fixed: Captions appeared where the user first looked, and they remained in that location throughout the experience. But this option was not favored, as participants felt they either missed the captions or missed out on the video when trying to locate the captions.
BBC’s innovative solutions are a step in the right direction. If we want to make these new, exciting technologies more universal, then developers need to seriously consider accessibility.
Through the BBC study, BBC found that there wasn’t a unanimous vote on which solution was the best, highlighting the importance of testing all solutions to find the most optimal captioning method. In addition, while one captioning method may be optimal for one video or program, it may not be optimal for another.
Ultimately, captioning and accessibility in 360° video and virtual reality need to be incorporated into the design workflow. That way, captions can be incorporated as an enhancement to the immersion.
Deep Dive on Major Web Accessibility Laws
There are about one billion people, or 15% of the global population, living with some form of disability who directly benefit from accessibility laws. According to The World Bank, individuals with disabilities are more likely to experience fewer opportunities for employment, higher…
Video Translation in a Hybrid World
After an unprecedented global pandemic that led to relying solely on digital communications, many people are slowly making their way back to in-person settings. Even as physical gatherings become more commonplace, however, we’re seeing many virtual components stick around in the form…
How to Handle Live Closed Captioning – and the Challenges
Technological innovation has paved a new way to conduct business, education, and life in general – particularly in a world forced to adapt to virtual substitutes during the pandemic. Most of the time, the technology we use is very helpful. For example,…