Get a Demo

Cielo24 Accuracy Analysis

We submitted files to Cielo24 in order to measure their true accuracy and provide empirical evidence of their product quality.

Read on for the results

We submitted 7 files with good quality audio, of varied subject matters, and with a duration of 5+ minutes to Cielo24.

What We MEasured

Spelling and Grammatical Errors

Since punctuation errors are subjective, we measured accuracy for both including and excluding punctuation and case.

Word Error Rate (WER)

The WER measures the number of substitutions, deletions, and insertions, divided by the number of words in the reference.

Average Accuracy

Cielo24’s average accuracy rate for the 7 files submitted ranged between 84.85% to 91.97% accuracy.

The industry standard for caption quality is a 99% accuracy rate. This means that there is a 1% chance of error or leniency of 15 errors total per 1,500 words.

The FCC, DCMP, and WCAG have all set captioning quality standards to ensure captions are comprehensible. Failure to uphold these standards could affect your brand, messaging, or result in a costly lawsuit.

Learn About Best Practice for Caption Quality
Cielo24's average accuracy rate for the 7 files submitted ranged between 84.85% to 91.97% accuracy.

File Comparison

Here’s a comparison of the files we submitted alongside their accuracy rate.

Cielo24’s average accuracy rate for the 7 files submitted: 84.85%

Even when we omit punctuation, Cielo24’s accuracy falls short of 99%, and the average accuracy rate for the 7 files comes to: 91.97%

*Why include or omit punctuation? Punctuation errors can be subjective, so we measured both cases. At 3Play Media, our accuracy rate is measured including punctuation.

Common Errors

Throughout files, we saw errors such as extra and incorrect words inserted into sections with no dialogue, false starts, misplaced commas, and spelling errors.

Incorrect Words

Cielo’s Errors:

  • Incorrect words
  • Extra words
  • Missed words
  • False starts

Why This Matters:

  • Impacts reading comprehension and retention
  • Incorrect wording creates confusion for the reader

Spelling

Cielo’s Errors:

  • Improper tense use
  • Incorrect spelling

Why This Matters:

  • Affects your branding
  • Affects the meaning of the content
  • Spelling errors distract the reader
mistakes include spelling center like centr
Mistakes include: Incorrect use of periods Incorrect use of commas Incorrect use of apostrophes Incorrect capitalization

Punctuation

Cielo’s Errors:

  • Incorrect use of periods
  • Incorrect use of commas
  • Incorrect use of apostrophes
  • Incorrect capitalization

Why Punctuation Matters:

  • Affects meaning of the content
  • Affects reading comprehension

Why Are There Errors?

Cielo24’s transcription process largely attributes to the inconsistencies and errors found within each file.

  • Each individual file that comes through is cut up into smaller segments.
  • These segments are then distributed across a pool of transcribers.
  • Once the transcription is complete, Cielo24 pieces back together the original file.

Note: There is no quality assurance process. The Final file is never reviewed before being sent back to the customer.

As a result of Cielo24’s process the customer must spend more time and money reviewing and fixing the inconsistencies of the final file.

a single file is split among multiple transcribers, then put back together

The Bottom Line On Quality

Cielo24 does not guarantee 99% accuracy.

Cielo24’s process and lack of QA lead to inconsistencies throughout a single file. As a result, quality is sacrificed.

onboarding icon

Get Started

Don’t Let Inaccurate Captions Hurt Your Content

3Play Media guarantees a 99% accuracy rate for every file. (In fact, we have a measured accuracy rate of 99.6%!)

View 3Play Media Plans
Talk to a Rep

Related Resources