The State of Closed Captioning in Higher Education [TRANSCRIPT]
LILY BOND: Welcome, everyone, and thank you for joining this webinar, entitled “The State of Closed Captioning in Higher Education.” I’m Lily Bond from 3Play Media, and I’ll be presenting today alongside Dr. Katie Linder, who is the research director at Oregon State University’s Ecampus research unit. We’re covering three different studies in this presentation. Two are IRB-approved research studies conducted by OSU in collaboration with 3Play Media. And the final one is a survey put out by 3Play Media.
Both surveys that Katie will be covering are available for download on our website. And the survey that I’ll cover will be available later this spring. And everyone who joins today will receive a copy when it’s published. We have about 45 minutes for this presentation, followed by 15 minutes for questions. And Katie’s going to start us off with the presentation.
KATIE LINDER: All right. Thanks, Lily. Hello, everyone. Today we’re going to cover a range of things from the three studies that Lily mentioned. We’re going to talk about how and why students are using closed captioning, some of the perceived benefits of captioning beyond accessibility, how colleges and universities are handling closed captioning, some current successes and limitations of captioning in higher education. We’re going to look at a comparison of captioning practices for face-to-face online and institutional marketing content at different institutions of higher ed. And we’re also going to look at how colleges and universities perceive and react to legal requirements for captioning.
As Lily said, for the two studies I’m going to be talking about, we have those study reports available via 3Play Media. And due to our time limitations today, I’m not going to go into detail about respondent demographics for those studies. But the context for that is all included in the reports, if you want to look at that in a little more detail. So let’s get started with how and why students are using closed captioning.
For the student survey that we did, we found out that almost 100% of students have courses that include at least some video. Of all of our student respondents, 99.7% of them have at least one course that includes at least one video. So we’re starting to see videos more and more in our higher education environments.
We found that overall, we had about 35% of respondents who noted that they always or often use closed captions when they’re available. And in response to the same question that we asked about– common use of captions– we found out 26% said that they never use them. We also had a little over 1,500 students who did not self-identify as having difficulty with hearing who responded to our study. And a little under 71% of that group uses closed captioning at least some of the time. So one of the important things that came out of our study was finding out that students are using captioning pretty significantly beyond accessibility reasons.
The percentage of respondents who always or often used closed captions in our study when they were available was [AUDIO OUT] for a really broad range of student groups. The original number that used closed captioning overall, always or often, just to remind you, was about 35%. So students with other disabilities, ESL students, students registered with an office of disability services, students with academic accommodations, and students who have difficulty with hearing were all higher in terms of how frequently those groups always or often used closed captions.
We wanted to know why students are using closed captions. And the top three reasons were that they help students to focus, they help them to retain information, and they help them to overcome poor audio quality. This is taken from some of our qualitative data, where we had about 150 comments from students asking them why they use closed captions. You’ll also see some other categories here related to things like environments. They need closed captions in quiet environments like libraries. Or maybe there’s other reasons why the video is difficult for them, such as their professor has an accent.
We were very interested in learning more about the perceived benefits of captioning beyond accessibility. And we found that there were five main themes that emerged, primarily from our qualitative data in our student survey, about why students find closed captions to be beneficial. One of those reasons was environmental, like I mentioned– those quiet environments.
They also found that video/audio quality was often poor if sometimes it was too quiet or sometimes the video was really old, and so the audio quality was just very difficult to understand. Or instructors were difficult to understand because they spoke too fast or they had an accent. Students also cited convenience frequently. They said that closed captions were faster and that it saved time to very quickly scan through a video’s captions to find a certain part if you wanted to rewatch again.
Several students, of course, mentioned accommodation. But the accommodation was not always related to hearing impairment. We also saw things like dyslexia and other kinds of print disabilities, that closed captions were helpful to students. And we also had many students who talked about closed captioning as a learning aid. And I’ll go into that in a little bit more detail.
There were a little over 1,200 qualitative comments that were included in this particular analysis. So some of the kinds of things that students said were that for environmental, “I don’t like playing videos with sound when I’m in an environment where doing so would be disruptive.” For video/audio quality, they said things like, “It helps when a video has poor sound quality.” For instructors being difficult to understand, they said things like, “My professor has a slight accent that makes her words fast and jumbled.”
For convenience, they said things like, “I find it easier to read along with the videos rather than sit there and listen to the professor talk.” And for accommodation, we had comments such as, “I’m dyslexic, so it helps me to know that the notes I’m writing down are both spelled correctly and in the right syntax.”
For the learning benefits, this was a significant component of student responses. And we had about 75% of student qualitative comments that talked specifically about learning benefits. So we broke this down even further into four categories– accuracy, comprehension, retention, and engagement.
For accuracy, students said things like, “If I need to write down notes, I know exactly how to quote a video.” For comprehension, they said things like, “It can help me deeply understand the lecture.” For retention, “By simultaneously reading and listening to the content, I am able to retain the information better.” And for engagement, they said, “They help me focus on the video instead of just tuning out the noise.” So we were very excited to see such a range of learning benefits that were coming out of closed caption use. And again, we had a pretty significant portion of this respondent group who are looking and using closed captions, and they are not having hearing impairments or hearing-related disabilities.
We also wanted to know about the helpfulness of closed captions more generally. And so as a percentage of the whole, we found that closed captions were perceived as helpful with 59.1% of respondents, noting that they were very or extremely helpful to them. But overall, 98.6% of student respondents found closed captions to be helpful.
The percentage of respondents who stated that closed captions were either very or extremely helpful to them was also higher for a significant number of student groups. So again, just to remind you, the original number for the group as a whole was about 59%. So the groups that were higher than this were students with learning disabilities, adult learners, students who have difficulty with vision, students who always or often have trouble maintaining focus, first generation students, students who have difficulty with visual representations, Pell-eligible students, students with other disabilities, students registered with an office of disability services, ESL students, students receiving academic accommodation. And not surprisingly, the highest number was students who have difficulty with hearing. But again, we ended up having about 12 different subgroups who found the helpfulness of closed captions to be higher than the average for those particular groups.
And a second study on institutions– we wanted to know how colleges and universities are handling closed captions because we know it can be something that’s pretty challenging for institutions. First, we asked about video creation guidelines. We wanted to know if anyone had institutional guidelines that existed to inform the creation of videos, just in general, and that were used at their institution. And we broke this out for face-to-face, online, and institutional.
And what you will see is the most likely place to have video creation guidelines was in the online setting. And we have the highest number of institutions that said “yes” there. But we also saw a significant number of institutions say, no, that there were no institutional guidelines that existed to inform the creation of videos to be used at their institution. You’ll also see a pretty significant number of people said they weren’t sure.
We also wanted to know if institutions track or inventory, in some way, the videos that they create. And again, you’ll see a significant number of institutions say that they don’t. And this, as you’ll see later in the presentation, makes it pretty difficult to do any kind of compliance check, because if you don’t know how many videos you have, you don’t know which ones are being captioned and which ones are not. You’ll also see here a pretty significant number of people saying that they’re unsure. An equal number of people said they were unsure in the face-to-face and online environments than those who said, yes, they do some kind of video inventory.
We asked about who can post videos at different institutions. And we saw in most cases, it was designated offices only. You’ll see that there were no cases where students could post videos just openly– any students. But frequently, there were designated individuals or designated offices, about a pretty good number of people also saying any faculty member or any staff member is allowed to do this. In our other category, we had a lot of people talk about marketing and that marketing and a public relations group were the main people who posted videos on behalf of the university. But for things like courses, obviously faculty may be posting videos more frequently.
We asked generally how much captioning is being done. And we found that for 87% of the institutions in the survey, closed captioning is created at some point for some videos. We also found just a very small set of institutions who said, no, and then some who also said they weren’t sure. But the vast majority said, yes– it is being done at some point for some videos.
We did want to know to what extent this was happening. And so we asked, for how many videos is closed captioning being created? And you’ll see the majority here, for both face-to-face and online, is that it’s happening some of the time. For some videos, closed captioning is being created.
The next level of responses that we got was people don’t know. They’re really not sure. And you’ll see in the case of all videos, how many institutions have all of their videos closed captioned. It’s a relatively small subset of the group.
One of the interesting things here is that it’s more likely for face-to-face videos to be captioned in terms of some of the time than it is for online videos. But you’ll also see that in the “most” category, online is trumping. So it’s definitely different in terms of what the modality is that the video is going to be shared in.
We also wanted to know if the approach– including the process, the staff, or the criteria for the videos that’s being used to create captioning– is the same, regardless of whether or not the video is produced for face-to-face classroom or for online or for institutional purposes. And you’ll see here that 40% of the institutions said no. So a good chunk of the institutions have different processes– and that might include staff and criteria being used– for creating the closed captioning for the videos, depending on the purpose the video is being created for.
We also asked if closed captioning efforts are centralized or decentralized. And we found– again, a large proportion of campuses have no aspects of captioning that are centralized, a little under 43% here. For some people, they had a centralized policy, a little over a quarter; a centralized operations team, a little under a quarter of schools; and a centralized budget, about one in five had a centralized budget for captioning. We had a small number who weren’t sure. So again, spread out in different ways about what centralization looks like in terms of the closed captioning process.
We also asked if closed captioning efforts are systematic and just when people think about their institution as a whole, how would they think about the efforts in terms of closed captioning. And we had over 50% say that they’re not at all systematic, and another 40% that they’re somewhat systematic or that they’re just systematic. We had very low numbers of people say, very systematic, and that they’ve really worked out what’s going on in their campus in terms of systemizing their processes.
We wanted to know how videos are prioritized for captions. And we’re going to talk a little bit later in the presentation about the confidence levels of campuses in terms of understanding requirements to caption. But one of the interesting things here is you’ll see that there’s only five campuses that talk about all videos being captioned, so about 10%, a little over 10%.
But the main reasons that people are prioritizing captions are by request, at over 50%, and based on whether there exists a need to create equivalent access. So the creation of captions are still very much tied to accommodation and disability. But you’ll also see in a third level– about 30%, 31%, 32% said that it’s based off of the place of where the video is being used. And some of the decisions about prioritizing for captions are based on things like medium. About the same number of people said it’s based on whether the videos are public or only visible to certain students. So there’s some really interesting thought processes happening here about which videos to create captioning for and which to prioritize.
We also asked if criteria has been articulated for what constitutes effective closed captioning. And we found in about a little under half the cases, this answer was, yes, there has been criteria that’s been created. But in over 50% of the cases, the answer was, no, or that people were not sure who were responding to the survey.
And this is very interesting because in our student survey, we found very specific things that students consider hindrances for closed captions, including closed captioning blocking important information, closed captioning not being synced properly with video/audio, and also containing typos or incorrect information in the captioning. And this was very much a hindrance for students. And all of those kind of hindrances are fixable. And they’re really tied to quality assurance measures. So this was really interesting to see. Many campuses have not articulated what constitutes effective closed captioning.
All right, so let’s talk about some of the current successes and limitations of captioning in higher education. We wanted to know why are institutions captioning, and specifically, is it related to a campus-wide initiative. And we found that almost a quarter of our campuses that responded have a campus-wide initiative around closed captioning, which was really interesting and something that I think we’d want to learn more about. But as you’ll see here, about 3/4 said no– they don’t have a campus-wide initiative, or they’re unsure.
When we asked about specific reasons why institutions are captioning, you’ll see that the top two are about compliance. They want to be in compliance with the law. And they want to avoid potential litigation. We had over 3/4 of institutions respond as their top choice. You’ll also see accommodation requests are coming in at third place right here.
The other interesting thing about this is number four is to support the needs of all learners, at a little over 60%. And there’s also, third from the bottom, in response to student demand that is not accommodation-related. So this is something that I think the student report and the student data that we collected is really bolstering both of those, that we know that students who are not requiring closed captions for accommodation are using closed captions as learning aids. And this may be a reason for campuses to think about utilizing those closed captions in broader ways.
We also asked why are institutions not captioning. And you’ll see the top reasons were just a lack of general awareness, not being sure that they needed to or which videos to caption. Many institutions talked about not having a budget or not having the staff or being unclear whose responsibility it is to provide the captioning.
And this actually supports an earlier study that I was involved with, that we produced an article out of in 2015. The study took place in 2013. It was a national study. And we produced an article called “Whose Job is It?” because we found in our national study that was related to online accessibility initiatives, people were very unclear whose responsibility it was and if it should fall to disability services or if it should fall to academic affairs or student affairs. And so it resulted in a pass the ball around. And because no one’s really responsible, certain things were not getting done.
So we see that again here, specifically related to closed captioning. But you’ll also see one of the things that is kind of interesting is the expensiveness, or the cost of closed captioning, is falling relatively low on the list. It’s definitely not in the top three or five. The budget piece is definitely number two.
But people are, I think, finding solutions that work for their budgets. I think it’s also important to note that nobody said that this is not a genuine need at our institution. Everybody in the study recognized the importance of closed captioning, even if they weren’t quite sure how to do it or they weren’t quite sure how to make it happen.
We also asked if institutions are monitoring closed captioning compliance. And you’ll see a little over half of them said no. Now, I’ll remind you– go back a couple of slides here– that institutions are captioning– the top two reasons, to be in compliance with the law and to avoid potential litigation.
But when we asked about monitoring that compliance, the majority of schools said that they were not doing it or they were not sure if they were doing it. So over half of institutions aren’t monitoring compliance, despite compliance being their top reason for closed captioning implementation. And this also relates back to our earlier slide about whether or not institutions are inventorying their videos. And in many cases, they’re not, which can really hamper this possibility of monitoring the compliance.
We wanted to know who ensures the quality of captions. And you’ll see that it’s a little bit across the board. In some cases– in a small number, about 15%, no one was designated to monitor the quality.
But we also know, in many cases, as we saw before, no criteria for quality has been established. Less than 50% of the institutions in this study knew of criteria for effective closed captioning. So this may be that offices are assigned this job, but they’re not necessarily given specific criteria of what it means. Perhaps not surprisingly, the top unit listed here was an office of disability services. A little under 50% of institutions said this office or unit ensures the quality of captions.
To offer a little bit more of a comparison of captioning practices for face-to-face, online, and institutional marketing content, we asked more specifically about formal guidelines existing to inform the closed captioning of videos in those units. And you’ll see that there’s pretty much equal numbers for face-to-face and online, in terms of answers of “yes” to that question, if formal guidelines exist, and also answers of “no,” that they don’t exist. You’ll also see that there’s several schools that were not sure. So this is another one of those areas where it’s kind of spread out a little bit across the board.
We wanted to know how closed caption guidelines systematically are communicated to potential video creators. And you’ll see, again, a relatively small number of people saying, yes. It was most likely to happen in face-to-face classroom video creation. And it was most likely to not happen with online, which again, was kind of surprising.
We asked who decides on individuals’ campuses whether to create captions. And you’ll see that this is, again, a little bit all over the map on different campuses. It was pretty common for an office of disability services to decide whether or not to create captions for face-to-face or online. You’ll also see high numbers of faculty involvement in those areas.
But you’ll see involvement from offices of instructional technology, the provost’s office, the VP of student affairs office, the CIO. The chief diversity officer was the lowest ranked here. Deans or chairs– students had some engagement in this. And then we had, again, some falling in “I don’t know” and “Other.” For Other categories, we saw more people being listed, such as centers for teaching and learning or online or distance education offices or instructional designers. That is what fell heavily into that category.
We also wanted to know who decides which caption solution to use, whether that might be internally creating those captions or outsourcing captions. And again, we saw high numbers from our office of disability services, from faculty. And then that other category, again, included a lot of online and distance education offices or instructional designers. And also, offices of instructional technology were also heavily involved here. But you can see that there’s a pretty big range of stakeholders on some campuses who are being included in this decision making.
We also wanted to know who creates the captions. And you’ll see that in many cases, it was the creator of the video. Many cases are also including a third party, external to the institution, or an office of disability services.
But here, we also saw a pretty broad range of who’s creating the captions. And because we know of the quality assurance issues going on with some of the hindrances with closed captions, it’s interesting to see that the creator of the video is, in a large number of cases, creating those captions but not all the time. And so many times captions are being created by people who are not involved necessarily with the creation of the video.
We also wanted to know how colleges and universities perceive and react to legal requirements for captioning. And so we included a statement talking about this in the survey, to level set, a little bit, expectations and to make sure that people knew exactly what we were talking about. We offered a definition for closed caption. And we also talked about federal disability law. And we asked if colleges and universities are confident that they understand what it means for their institution to be in compliance with these laws. And we had 38, or about 80%, say they’re confident or very confident that they know what it means to be in compliance with both federal and state accessibility laws regarding closed captioning.
When we asked what best described their understanding of what it means for their institution to be in compliance with federal and state laws, we saw a range of responses. About 50% identified captioning all videos as what it means to be in compliance. And it was important to note that nobody said they’re not legally required to provide any closed captions. But I would imagine because, in part, the state laws are different across different institutions, this is an area where we’re seeing a wide range of people, in terms of what best describes their understanding. But again, a large portion of them feel that they do understand what compliance means.
We asked if they feel they are meeting the legal requirements for captioning, based on their understanding of what it means. And almost 80% believe that they are only partially meeting requirements. There’s about 2% that said they’re exceeding and about 15% that said they’re meeting. But a large– over 3/4 of institutions said they were only partially meeting the requirements that they’re aware of.
Another important thing that came out of this study is we asked about proactive and reactive responses to closed captioning efforts. And we found that almost 2/3 of institutions said that their response to closed captioning implementation is either more reactive than proactive, or it’s primarily reactive. It’s designed for accessibility and is done often only after accommodations are requested. So this was something that we thought we knew, and it was backed up in the study, that, again, almost 2/3 are having more reactive responses to closed captioning than doing it in a more proactive way.
So I’m going to go ahead and hand the presentation over to Lily, and she’s going to give a little more information about the survey created by 3Play.
LILY BOND: Thanks, Katie. And as you said, I’m going to shift gears a little bit and speak about “The State of Captioning” Survey, which 3Play Media put out at the end of last year to get a better gauge on the current state of the industry of captioning in higher education. The full report will be released later this spring, but we’re going to take a first look at the data now. So we reached out by email to just over 18,000 institutional representatives in higher education, who were asked to complete a 53-question online survey. And we got 832 respondents from higher education, and we’re going to look at their responses now.
So we have a pretty diverse range of respondents, in terms of student enrollment. About 58% have enrollments between a thousand students and 20,000 students. But we have some smaller and larger institutions in there as well. And about 75% of respondents were from public colleges and universities. And about 25% were from private institutions.
We got a broad range of respondents across the US, pretty evenly distributed across mid-Atlantic, Midwest, New England, South, Southwest, and West. We wanted to get a sense, first, for how much video was being produced or published so that we could understand their responses to captioning a little bit better. So most of the respondents are publishing between 11 and 100 hours, or they don’t produce video in-house. And those people are publishing third-party video or are purchasing video from other organizations.
So to break down by type of video, we wanted to know who was talking about live video versus recorded video. Most of them were talking about recorded video. A lot of them were producing both recorded and live. And then a good number were also purchasing video.
And where we see a real change is where– how many people caption these different types of videos. So on the bottom left, the chart there shows that there’s a really strong correlation between recorded video and captioning. But more people than not do not caption their lives videos.
And then it’s interesting to look at purchased video. It’s a good look at what vendors are providing in terms of accessible video. Most people said that those videos are captioned only sometimes. Many said “rarely” or “no.” And about 65 said that they are always captioned.
So we wanted to look at whether or not these institutions have a centralized process for captioning. And only 35% said that captioning is centralized at their organization. But another 40% or so did say that it’s on their roadmap, or they’re establishing a centralized process now.
One thing that we saw pretty universally is that captioning needs are on the rise. We asked how they foresee their captioning needs changing in the next year. And you can see most see them increasing significantly. And then many also see it increasing moderately. Fewer see it staying the same. And only a few said that they see their needs decreasing.
Budget always comes into question when it comes to captioning. So we asked, what is your annual budget for captioning? And you can see that just over 50% were not sure about their budget. But of all respondents, 12% who caption their videos said that they have zero budget for captioning.
And when you look at a percentage of how many people who do know their budget have a $0 budget, that actually goes up to 25% of respondents. So 25% of people who do know what their budget for captioning is have a $0 budget. So that’s amazing. And it definitely speaks to the biggest barrier that we often see, which is budget.
So we wanted to do a deep dive on caption quality, because quality means a lot of different things to different people. And compliance is really about creating an equal experience. So we wanted to see what people’s perception of that quality was.
We asked first if these institutions had received feedback about the quality of their captions. So about a third of respondents have received feedback about the quality of their captions. And then when we asked what that feedback was, about 86% said that the feedback was that the captions are generally high quality, with some inaccuracies, or are consistently high quality. So that’s a pretty good number. Obviously, we want to see those numbers higher.
But we wanted to dive a little deeper into that. So we then asked, on a scale of 1 to 10, how would you rate your caption quality? And you can see that most people rated their caption quality at an 8. But we have a lot of 7s, 9s, and 10s as well.
We then wanted to know whether or not these institutions are reviewing or editing their captions. And what we found was that the vast majority always or sometimes review their captions. 40% say they always review their captions. But the table flips when you look at editing. So on that side, the vast majority rarely or only sometimes edit their captions.
The big question that we often hear is, are automatic captions good enough? So we wanted to know how many people are using automatic captions for their captioning purposes. And here, we see that 21% say they use automatic captions for all or some of their captioning needs. And then you see a good chunk, about a third, say that they start with automatic captions and then edit them.
And then one note of comparison is that on the last slide, we saw that only 15% of respondents often or always edit their captions. And then we’re seeing this giant chunk of people who say that they edit their automatic captions. So there’s a little bit of a disconnect there.
So then we wanted to break down some of these categories and compare a couple of different factors against each other. So we wanted to see, do respondents who rate their captions a 9 or 10 review or edit their captions? So the vast majority here say that they always or sometimes review their captions. So we’re thinking that they should be accurate about that 9 or 10 quality gauge. And then most of them rarely or never edit their captions, which is interesting. So almost all of the people who say they have 9 or 10 quality captions are reviewing them but not editing them.
Then we looked at, how did people who said that their captions were generally high quality or consistently high quality rate their caption quality on a scale of 1 to 10? And here, people who said that they have generally high quality– only 45% of them rated their captions at a 9 or a 10. And for people who said that they have consistently high quality captions, only 84% said 9 or 10.
Once you jump below a 9 out of 10 for caption quality, you’re really losing a lot of meaning. So the generally accepted standard for accurate captions is 99%. So once you’re getting down below the 9, you’re really looking at poor quality. And here, we have even people who said that they have high quality captions, but they rate their captions at a 5.
We also wanted to see if people who rated their captions at a 9 or 10 use automatic captions for all or some of their videos. And we were pretty surprised to find that 17% of people who rated their captions a 9 or a 10 use automatic captions. Automatic speech recognition is generally between 50% and 80% accurate. So there’s a huge disconnect here.
We also wanted to look more closely at people’s understanding of their legal obligations and requirements for captioning. So we started by asking whether or not respondents know which accessibility laws apply to them. And we found that 70% said that they’re confident that they understand which accessibility laws apply to them.
And then we asked, if yes, which laws apply to you? And the answers here are across the board, which is fine, because all of these laws could apply in different circumstances. But you’ll see that 43 people who said that they were confident they knew which accessibility laws applied to them then said they weren’t sure when they were given a choice to select the laws.
So then we wanted to know how confident people were that they understand what it means to be in compliance with the closed captioning requirements laid out by these accessibility laws. And we found that 65% said they were “pretty” or “very” confident that they know what it means to be in compliance.
So then we wanted to know– do people have a clear policy for captioning compliance? And we found that only 19% of respondents have a clear policy for captioning compliance, although you’ll see here, again, about 50% have captioning compliance policies on their roadmap, or they are establishing one now.
And so, similarly with quality, we wanted to break down these legal obligations and see how people were rating themselves compared to the way they were responding. So we asked, of the people who said that they were pretty or very confident that they understand captioning compliance, how many of those have a clear policy for captioning compliance? And we found that only 26% of them have a clear policy. So there’s definitely a disconnect there. And it grows as we look a little bit deeper.
So then we asked, how did respondents who said that they were pretty or very confident that they understood caption compliance rate their caption quality? And you’ll see that only 50% who are confident they understand what caption compliance means rate their caption quality at a 9 or a 10. So that’s pretty shocking.
And what’s even more shocking is that someone rated their captions a 1 and said that they understand what caption compliance means. And another person rated it a 3, and we’ve got five 4s there. So that’s a huge disconnect there.
And then we went a little bit further, and we said, how did respondents who stated that they had a clear policy for caption compliance rate their caption quality? And we see here only 66% rated their caption quality at a 9 or a 10. We still have that person who rated their captions at a 1, as well as two people rating them at a 5 and five people rating them at a 6.
So then we wanted to look and just ask some questions about institution size and how that affects budget and policy. So we mapped institution size by student enrollment to annual budget for captioning. And you’ll see that there’s a minor correlation here. You definitely see higher budgets more often in the larger schools. But there are still $0 budgets across the board. So we didn’t see a huge correlation here, although there is some.
Where we did see a big correlation is size of institution versus existence of a clear captioning compliance policy. You can see the lines here. The percentage of schools with a clear policy goes up for bigger schools. And the percent of schools with no policy goes down for bigger schools.
So that’s it for our data. A few takeaways here– most schools do not have a centralized procedure for captioning. Almost universally, institutions see captioning needs increasing by next year. And then we saw a lot of disconnects. So there’s a large disconnect between how people rate their caption quality and their understanding of compliance or what high quality means.
And then there’s also a disconnect between respondents’ confidence in their understanding of accessibility laws and their corresponding caption quality and policies. We saw a slight correlation between institution size and captioning budget and then a clear correlation between institution size and existence of a clear captioning policy.
So I know that was a lot of data. We’re going to open it up for Q&A. I want to encourage people to continue asking questions as we go through this. We are keeping track of them.
And I also wanted to make note– on the left-hand side of the screen here, you can see Katie’s and my contact information, as well as links to download the first two studies that Katie talked about. And then you can also find some more upcoming webinars with 3Play Media, which you can register for on our website.
So to kick off the Q&A, Katie, there are some questions about the demographics for the student and institutional study. Someone is asking, how many schools participated in the survey, and what was the breakout of self-identified students with disabilities versus non-self-identified? And also, who responded to the survey on behalf of the institutions?
KATIE LINDER: All right. These are great questions. So I do want to reiterate that all of this is in the report. So if you really want to get detailed breakdowns of who responded to each study– and also, all of the data tables of those demographics are also included at the end of each report. So you can dig really deep if you want to.
For the student study, we had 15 institutions who helped us recruit students. And we ended up using a little over 2,100 student participant responses in our analysis. And I’m looking just to see– we actually broke down disability in the study in a range of ways.
And so I would recommend going to the data tables, because we asked specifically about hearing impairments, visual impairments. We asked for people who struggle with visual elements, like tables and graphs. We asked about things like learning disabilities, and all of that is kind of broken down. But for the hearing impairment, specifically, I think we have 1,569 students who did not have a hearing impairment and who are responding to this study. So 1,569 out of a little over 2,100.
For the institutional survey, we had 47 institutions that we included in our analysis, in our responses. The majority of them were institutions that were PhD or professional degree-granting institutions. That was 21 of them. And then we had another 11 that graded bachelor’s and master’s, another eight for community colleges, and then a few other forms of schools.
In both studies, we had people coming from all over the United States. This is a study that we kept limited to the US, just specifically because we were asking about compliance of certain laws that were federal and state laws. I know there was another question in there, Lily. Which one did I miss?
LILY BOND: The other one was, who responded on behalf of the institutions?
KATIE LINDER: Thank you. OK, so we had sent out a call for this to three primary groups. And we asked those groups to liaise at their institutions to find out who would be the best person to respond to the survey. Or if they wanted to respond as a group, we offered that as an option as well, that people could get together and try to answer the questions as a group.
So we originally sent the information to disability services direct, to people who were heads of academic affairs, like a provost equivalent, and to people who were heads of student affairs, like a VP of student affairs equivalent. And we asked them to decide who they thought was the best person to answer the survey. And what we saw through the responses is a lot of people had knowledge for certain areas but not others. So for example, the people who had knowledge of classroom practices, either face-to-face or online, maybe struggled with knowledge about marketing purposes. And it may be just because of where their position was at the institution.
For anonymity reasons, because we were asking institutions to disclose, basically, whether or not and to what degree they were following federal law, we did not ask for anything that could make either the person responding or the institution identifiable. And so all those institutions answered the same survey.
And we were very careful about making sure that they were protected, especially if they were saying that they, for whatever reason, were not following the law. So we didn’t ask people to identify, for that reason, where they were coming from from the institution. And we put out a little bit of trust that the institutions would find us the best person to answer those questions if they chose to participate.
LILY BOND: Thanks, Katie. There’s another question here that I think both of us should answer, about the separate surveys. What data from the studies did you find most alarming and most surprising?
KATIE LINDER: That’s a good question. Do you want to start, Lily?
LILY BOND: Sure. So I mean, I just know off the top of my head– from our survey, I was shocked at how many people rated their caption quality at a 9 or a 10 but used automatic captions without editing them. That was just definitely eye-opening for me, because that means that these people believe that their automatic captions are high enough quality to accommodate a student who is deaf or hard of hearing. And that really stuck with me from our survey.
KATIE LINDER: I would say, for our survey– I wouldn’t necessarily call it alarming. I think it was something we already knew. But it was really important to see it reflected in the data. I guess it is alarming– and that is how much confusion there is in higher education around closed captioning.
And I think Lily’s survey also– the 3Play Media survey– also really emphasizes this. People don’t really understand which laws they need to be paying attention to. And in some cases, I think, there are some contradictions between the federal laws and the state laws. And it changes from institution to institution.
There’s a real lack of general understanding at an institutional level across different stakeholder groups about what closed captioning needs to do, how it needs to be implemented, who’s responsible it. There was just a lot of sense of people scrambling a little bit to figure this out. And I don’t know that that all falls on the shoulders of institutions. I think that the laws themselves are a little bit confusing.
So it is alarming that we have so many institutions that are struggling with this, especially given that we know large portions of students are using this as a learning aid. And that, I think, is one of the more exciting things that this study found on the student side, is we know that they’re using these things in a real range of ways and they’re getting really creative, actually, about how they’re using closed captions. So if we could make that tool more widely available, maybe it would create a support structure for student learning in a broader way.
LILY BOND: I totally agree, Katie, and that was a huge takeaway from us too, is just the vast confusion and yet the huge benefit of captioning. And one hope is that these studies will allow institutions to better organize themselves and really see the value in captioning and get it together to centralize a little bit more.
And along those lines, someone is asking, have these studies made a difference in how OSU approaches captioning?
KATIE LINDER: We get that question every time. I’m really glad people keep asking it.
LILY BOND: I know.
KATIE LINDER: So we actually ran the study internally first, and in particular with our campus students, with our students who are online. And so I can really speak mostly to that, because that’s the unit that I’m located in here at OSU. We’ve certainly shared these studies more broadly on our campus as well, with our larger Office of Disability Services and Disability Access Services.
But here at ECampus, there were very particular things and some of them, kind of granular, that did change how we did certain kinds of captioning. And I’ll give just one example. So our multimedia group approached me and said, you know, it’s kind of hard for us to put a closed captioning icon and a button for it on some of our multimedia, on every page. And would it be enough if we just put a closed captioning icon on the front, like in the very beginning, and students could choose to have it on or off for the duration of looking at that multimedia?
And I said, based on our study, it needs to be on every page, because we know that from the study, students turn it on and off, because maybe they don’t know how a certain word is spelled, or there’s a certain part of video or the audio that they can’t understand, or they move from a quiet space to a loud space. And so they’re not just turning it on in the beginning and using it the entire way through.
And so that was something that made an immediate impact on our multimedia team when they were designing some of their products for our courses, was to make sure that we put a closed captioning button on every page and that students could toggle it on and off, because we know that they’re using it in a range of ways. So it’s starting to have impacts. But we’re also just getting to the point where we’re distributing information from the study to different groups, because there’s different stakeholder groups that might be impacted in different ways based on each individual study and the kinds of information that’s coming out of it.
LILY BOND: Thanks, Katie. Another question here from the institutional study– who’s covering the cost of closed captioning in these schools that were surveyed?
KATIE LINDER: Good question. So I am going to do a very quick glance through the study to see if I can get to the part about budgets, because there is a section on budgets that we did not talk about today. And I would refer you to the study to talk about that.
Let’s see. So we asked about where the budget was housed in different campuses. And we also asked about the amount of the budget. And what we found was– I’m sorry I’m kind of scrolling through as I go to see if I can find this information on the fly.
We actually found that in a lot of cases, the budget was spread across a range of offices. One in five respondents noted that the budget for closed captioning is located in more than one office or unit. The largest number of respondents had said that closed captioning budgets were housed in a disability services office, but that was just 12 of our 47 respondents. So it’s really spread out in a range of ways across institutions. And I would refer you to the report if you want more specifics on that.
LILY BOND: Thanks, Katie. Someone else is asking, do you know if there were certain subjects where students found captions to be most helpful?
KATIE LINDER: So we did not break this out in that way. However, we had many qualitative comments, which perhaps, not surprisingly, discussed the importance of captioning in foreign language courses, and not subtitles, not captions that were showing the English equivalent to a foreign language, but just the closed captions that showed how, for example, a foreign language word was spelled or how a sentence was constructed. And so students really liked to see the visual of the words as they were hearing the words, and they felt like that was helpful to their learning. So that was a one that stuck out, that we had a lot of students talking about using closed captioning in foreign language courses.
LILY BOND: Thanks, Katie. Someone else is asking a general question– do you have any suggestions for how to encourage commercial video producers to add captions or subtitles for the deaf and hard of hearing to their products? We’re having difficulty finding captioned versions of videos we want to purchase for our library.
That’s a great question. And I think the best advice that I have is to include accessibility in your contracts with these organizations and these vendors. And if possible, get a VPAT from them, which is a Voluntary Product Accessibility Template. The more that you can write accessibility into your contracts with these vendors, the more it’s going to encourage them to caption more videos and to really put some of the impetus on them, because otherwise, you, the institution, is liable. The vendor is not.
Katie, I don’t know if you have anything to add to that.
KATIE LINDER: That was exactly what I was going to say is I think the more we can pressure both publishers and vendors to really understand that this is not optional and that for our kind of compliance issues in higher education, this is a mandatory thing that we need from them. And we show them with what we buy. And we just refuse to buy certain things.
But I have also seen the challenge of doing that, having a procurement policy like that on a campus. And it requires all the stakeholders involved in that kind of purchasing to have that knowledge and to have across the board understanding of that procurement policy and why it’s there. And so I would say another possibility– and actually, a lot of institutions have contacted me about this– they want to share their reports with other people at their institutions so people have a broader understanding of what’s going on and in particular, how closed captioning helps students as a learning aid.
And so I absolutely recommend also sharing the reports. You could share the student report, in particular, with vendors to say, this is why we’re doing this. In addition to the compliance, we know it helps students learn.
LILY BOND: Thanks, Katie. Someone else is asking, were the institutions asked about copyright laws or how many captions were being created for copyrighted materials? We did not ask that in the 3Play Media survey. Katie, was that asked in the institutional survey?
KATIE LINDER: It was not asked. And it’s definitely an area for further study, because I think that when it comes to closed captioning, that’s one of the most common questions that gets asked, is questions around copyright and concerns about what it means to change copyrighted material to include closed captions. So I think that’s definitely an area that needs some further exploration.
LILY BOND: Thanks, Katie. Another question on the OSU study– was the data that you presented for instructional videos, institutional videos, like marketing or both?
KATIE LINDER: Was it for the institutional study or the student study?
LILY BOND: The institutional study.
KATIE LINDER: So for the institutional study, we broke it out in the survey as being face-to-face videos for classrooms that were students or meeting primarily face-to-face, for online videos, for online classrooms, and then in a third way, for marketing purposes. And we defined each of those in the survey as what did we mean by that.
And so for example, for marketing purposes, I think we said something like, for institutional promotional purposes, or alumni outreach– that kind of thing. So we did differentiate that. And that’s some of the data that you saw presented today, that was broken up by face-to-face, online, and institutional purposes.
LILY BOND: Thanks, Katie. There are a couple of people asking questions about their legal requirements and where they can find out which laws apply to them. We have a lot of resources on our site, at 3playmedia.com, including an entire section on accessibility laws and a quiz called “Which accessibility laws apply to you,” which can help break out the specific requirements based on your unique needs and requirements as an institution. We can include some of those links in the follow-up email.
KATIE LINDER: Lily, you also– I just want to point out. You guys have a very good white paper that outlines the laws and outlines some of the confusion around them. So I don’t know if we can link to that. But that’s one of the ones that I always refer people to.
LILY BOND: That’s very nice of you, Katie. And yes, we can definitely link to that white paper in the email as well. And I think we have time for one more question. Someone is asking, do you have any examples or resources for creating compliance policies? Katie, do you have anything on that?
KATIE LINDER: Wow, that’s a really good question. My immediate answer is no, but what I would point to you, too, as maybe a possible resource for that, which you may already be very familiar with is AHEAD, the National Organization for Disability Services administrators across higher ed. I would imagine there may be some resources there.
I also have a couple people I know that have done that but I don’t want to just give their names out here. I’d want to ask their permission first. But feel free to email me with the contact information that’s on the slide. And I can maybe put you in touch with some people that I know have worked on this pretty in-depth.
LILY BOND: Thanks, Katie. And I would also say that one of the best resources is collaboration with other institutions and learning from institutions that are really far ahead in captioning and accessibility. And most of them have their policies available freely to the public on their websites. If you just look at their accessibility link right on their home page, that usually gives some good examples of schools that are doing it right. And we also have a lot of webinars with different schools and how they’ve developed their policies, which you can certainly use those resources as well.
And that’s about all we have time for. Thank you, everyone, so much for joining. And Katie, thank you again for a wonderful presentation. It’s always nice having you on the line.
KATIE LINDER: Thanks, Lily, it was really fun to be here.
LILY BOND: A reminder to everyone that you will receive an email tomorrow with a link to view the recording and the slide deck. And I hope that everyone has a wonderful rest of the day.