Verbit Accuracy Analysis

Verbit says their captions are 99.9% accurate. We submitted files to Verbit in order to measure their true accuracy and provide empirical evidence of their product quality. 

What We Measured

Spelling and Grammatical Errors

Word Error Rate (WER)

The WER measures the number of substitutions,  deletions, and insertions, divided by the number of words in the reference.

Order & Delivery Process

We measured the time elapsed from ordering to order confirmation and time elapsed from order received to delivery.

Common Errors

Throughout files, we saw errors such as extra and incorrect words inserted into sections with no dialogue, false starts, misplaced commas, and spelling errors.

Spelling

Incorrect Word

Missing Speaker Identifications

Read on for the results.

Revision and Support Experience

We also measured the time elapsed for editing and correcting the returned transcripts from Verbit. 

Since punctuation errors are subjective, we measured accuracy for both including and excluding punctuation and case.

We submitted 8 files with good quality audio, of varied subject matters, and with a duration of 5+ minutes.

We submitted 5 files with good quality audio, of varied subject matters, and with a duration of 2+ minutes  to Verbit. 

"the Capitolini"

"More older,"

"synapsis"

"200 people will be saved.

"linguists who uses the library"

Verbit's average accuracy rate for the 5 files submitted ranged between 94.83% to 95.58% accuracy

The industry standard for caption quality is a 99% accuracy rate. This means that there is a 1% chance of error or a leniency of 15 errors total per 1,500 words.

The FCC, DCMP, and WCAG have all set captioning quality standards to ensure captions are comprehensible. Failure to uphold these standards could affect your brand, messaging, or result in a costly lawsuit.

For more information on caption quality, read Best Practice for Caption Quality.

"all kinds of makeshifts"

"programmer with 

decisions"

"economical"

"Typing on"

"some kind of a plane"

"the Capitoline

"Moreover,"

"synapse"

"will be safe"

"linguists who use

"the Capitolini"

"More older,"

"synapsis"

"will be saved"

"linguists who uses"

Verbit demonstrates quick turnarounds and low-cost rates. However, based on our findings, Verbit does not demonstrate - nor can they guarantee - 99% accuracy.

The Bottom Line on Quality

© 3Play Media, Inc.

Verbit says their captions are 99.9% accurate. Based on this analysis, that is far from the case.

Verbit's Errors:

  • Incorrect words
  • Extra words
  • Missed words
  • False starts

"the Capitoline"

"Moreover,"

"synapse"

"200 people will be safe."

"linguists who use the library""

Correct Words

Verbit's Errors

Correct Words

Verbit's Errors:

  • Use of nonexistent words
  • Incorrect spelling of names and titles
  • Incorrect tense usage

Why spelling matters:

  • Affects your branding 
  • Affects the meaning of the content
  • Spelling errors distract the reader

Verbit's Error:

  • Missing speaker identifications 

Why speaker IDs matter:

GIULIO 

MAGLI:

Today,

we

explore

>

>

Today,

we

explore

"all kinds of mixtures"

"program our

decisions"

"ecunemical"

"Typing a word"

"some kind of a plinth"

Correct Words

Correct Transcript

Verbit's Error

Verbit's Errors

  • Helps distinguish speaker
  • Important when there are multiple speakers
  • Incorrect wording creates confusion for the reader
  • Impacts reading comprehension and retention

Why this matters:

Why this matters:

Verbit's Errors:

"all kinds of mixtures"

"program our decisions"

"ecumenical"

"Typing a word"

"some kind of a plinth"

"all kinds of makeshifts"

"programmer with decisions"

"economical"

"Typing on"

"some kind of a plane"

Correct Words

Verbit's Errors

"German Lothan Meyer"

"readen"

"overthrow"

"gonna"

"Penn Dragon"

"German Lothar Meyer"

"read"

"overthrew"

"going"

"Pendragon"

Correct Transcript

Verbit's Errors

>

>

Today,

we

explore

GIULIO

MAGLI:

Today,

we

explore

Punctuation

Verbit's Errors:

  • Missing commas
  • Missing apostrophes
  • Missing hyphens
  • Incorrect capitalization

Why punctuation matters:

  • Affects meaning of the content
  • Affects reading comprehension

"the Earth's surface"

"It's extremely peculiar"

"two-thirds probability"

"the sun passes"

"the earth surface"

"Its extremely peculiar"

"two third probability"

"the Sun passes"

Correct Words

Verbit's Errors

GET STARTED WITH 3PLAY MEDIA

Don't Let Inaccurate Captions Hurt Your Content

3Play Media guarantees a 99% accuracy rate for every file.

(In fact, we have a measured accuracy rate of 99.6%!)

Videos Accuracy Rate Including Punctuation Errors Accuracy Rate Omitting Punctuation Errors*
Video 1
Duration: 14 min, 18 sec
96.08% 96.58%
Video 2
Duration: 16 min, 27 sec
94.83% 95.65%
Video 3
Duration: 13 min, 04 sec
93.54% 95.87%
Video 4
Duration: 8 min, 18 sec
95.01% 95.40%
Video 5
Duration: 6 min, 46 sec
94.79% 95.51%

Here's a comparison of the files we submitted alongside their accuracy rate.

Verbit's average accuracy rate including punctuation for the 5 files submitted: 94.83%

Even when we omit punctuation, Verbit's accuracy falls short of 99%, and the average accuracy rate for the 5 files comes to:  95.88%

*Why include or omit punctuation? Punctuation errors can be subjective, so we measured both cases. 

At 3Play Media, our accuracy rate is measured including punctuation