National Research Results: How & Why Do Students Use Closed Captioning? [TRANSCRIPT]
LILY BOND: Welcome, everyone, and thank you for joining this webinar entitled “National Research Results– How & Why Do Students Use Closed Captioning?” I’m Lily Bond from 3Play Media, and I’ll be moderating today. And I’m joined by Dr. Katie Linder, who is the research director at Oregon State University Ecampus, and she’s also the head researcher on the study being presented today.
We have about an hour for the presentation. We’ll try to have the presentation itself be about 45 minutes and leave 15 minutes at the end for Q&A. And with that, I’m going to hand it off to Katie, who has a really great presentation prepared for you.
KATIE LINDER: Hello, everyone. Thank you so much for attending the webinar today. I’m really excited to share with you some of the results that we have from this national study, and I’ve been pleased to see the response to this webinar and how interested everyone is. I’m going to go ahead and get started. If you do have questions about particular slides, I encourage you to reference the slide number– that’s going to be in the lower left corner– when you ask your questions. And that way, if we have questions at the end, I can go back to a slide if needed. I’ve also included the different Twitter handles for the Research Unit here at Oregon State, my own Twitter handle, and also 3Play Media, if you’re going to be tweeting the presentation.
Today in this presentation, I’d like to give you a little bit about the context of the study. And then I’ll share quite a bit about our data and results, both the quantitative and the qualitative data that we collected. And I’m going to end with talking a little bit about some takeaways that we have already kind of thought through in terms of the results that we’ve had from this study. And then, of course, we’ll also have some Q&A at the end.
So this study grew in part out of another study that we had started at Oregon State University just looking at student use and perceptions of closed captions in fully online classrooms at Ecampus here at Oregon State. And I wanted to let you know that those results are posted to our website. We will provide a link in the chat window if you want to look at either one of those studies.
But as we were working on that study, we began to wonder what this would look like on a more national scale. And so we began to formulate this national study in fall 2015 as a collaboration between the Ecampus Research Unit and 3Play Media. We ended up using the survey instrument that we pilot tested with approximately 700 OSU Ecampus students, and we modified that instrument to also include some information about transcripts as well as captions.
So we ended up creating a two-part study, and it was a student survey on the use and perceptions of closed captions and also an institutional survey on solutions for closed captions. And the results from that second study will be discussed in a webinar on October 12. I’m going to talk about that a little bit later in the presentation.
The research questions that we focused on for the study were “To what extent are students aware of the availability of video closed captions and transcripts in their courses?” “To what extent do various student populations use video closed captions and transcripts?” “Why do students use video closed captions and transcripts?” “How are they using video closed captions and transcripts” and specifically to support their learning? And “To what extent do various student populations perceive the use of video closed captions as potentially valuable to their learning?”
When we were recruiting for this study, we sent out emails to a little over 3,700 institutional representatives announcing the study, and we also recruited via Twitter and word of mouth. We held a series of webinars describing the study. Many of you may have attended one of those. And then for institutions that chose to participate in the study, recruitment materials for the student survey were shared for distribution.
When students were consented for the survey, students were told that their institution was partnering with Oregon State University and 3Play Media, and students were asked to complete a 46-item online survey that included Likert scale questions but also some open response questions. Students had the option at the end of the survey to provide their email to be entered into a raffle for one of 50 $25 Amazon gift cards.
So the response that we received, we had 15 institutions who agreed to assist with recruitment, and from those 15 institutions we had about 2,800 students who responded. We did exclude from the study participants who were under the age of 18. And of all the different responses we received, we ended up including 2,124 in our analysis of the outcomes after we completed our data cleaning procedures.
I think a really important thing to note about this study is that we learned pretty early on that many institutions only think about closed captions in relation to student accommodation. I received a lot of questions from institutional representatives or even just comments via email that they might not be a good fit for our study, because they only had five students who were using closed captions at their institution. And this study was really about trying to find out how all students used closed captions, and that was something we really tried to emphasize in our messaging.
Even though that was something that was in our messaging, we found that some institutions only sent out the survey to students who were receiving captions for accommodation. And so you’ll see in our list of institutions that’s coming up some that had under 10 respondents, and that may be part of the reason, is that it was sent out to a relatively small sample on their campus. So just an important thing, but I think you’ll also see we have a pretty diverse pool of participants, and we were really excited by how many students responded.
To give you a sense of the participating schools in this study, about 20% of them were private institutions versus 80% that were public. 20% were two-year institutions, and 80% were four-year institutions. 34% were from the Midwest, 27% were from the West, and 13% each were from the Northeast, the Southeast and the Southwest. So we got a pretty broad distribution from across campuses, which was wonderful.
I want to share a little bit of context about the students who responded so you have a sense of who is telling us about their experiences with closed captions and transcripts. As you’ll see, the majority of students are juniors and seniors, almost half of them that responded. Which I thought was great, because we get a good sense of several years of their schooling and higher education. Almost 83% of respondents are undergraduate students, so it’s a large population of the sample that we had as well.
I’m going to share a bunch of numbers with you next. I encourage you to look at the slides if you want to review these afterwards. I’m going to go through them relatively quickly. There will also be a lot more information in the report that we’ve been creating that will share these numbers as well.
So about 64% were adult learners, and this is the students who we counted as over the age of 24. You’ll see also later on in the presentation we had a high number of learners respond who were primarily online students, so that may be in part why this adult learner number is so high. 88.7% cited English as their first language. And we also had 28% who identified as first generation.
Now, that number may actually be higher. We had several students who weren’t sure about their guardians’ or their parents’ high school graduation or other schooling. But this was kind of the best number we could come up with, given the data we had. 39% of the students who responded were Pell eligible. 67.7% identify as female, and 76.2% identify as white.
We also asked for a range of responses about students’ disability status or impairment status. So on the left side of the screen, you’ll see that we asked a question about how often students are struggling with maintaining attention in class, and 30.2% said always or often. Another 47.1% said sometimes. So we had over 75% of respondents mention that they struggled to some extent with maintaining attention in class. That’s perhaps not surprising to many of you who are listening.
We had 19.1% who said they had some kind of difficulty with hearing. Again, this number may be slightly higher, because of who was targeted at each institution for recruitment for this particular study. We had 10.8% who expressed difficulty with graphs and charts, 37.2% who had difficulty with vision, and 9.5% who noted they were diagnosed with some kind of learning disability.
In our student respondents, we had 10.7% who noted that they have some other kind of disability that we did not include in the options that they could check. 13.1% are registered with an Office of Disability Services, and 11.4% require some kind of academic accommodation.
We also asked, as I mentioned earlier, what kind of course modality our students are taking that were in the survey. And we found that a little over half are taking courses primarily online, and a little over one third are taking courses all face to face. So we have kind of a nice balance of students who are in fully online environments and also students who are in face-to-face environments.
One of the key metrics for this study is that students reported that 99.7% of their courses include some kind of video. So that is absolutely something that we were prepared for. But it’s nice to have that number to know the students are certainly engaging with video in significant amounts of their schooling.
So next, I’m going to share some of our results about the availability of closed captioning and students’ uses and perceptions of it. We asked students approximately how many videos in their courses had closed captions as an option, and we found there was quite a spread here. There was about 28.8% percent that said all, most, or many of their courses have closed captions, so less than a third. Important, I think, it’s also to note, that 27% were not sure. And we’ll get into some questions that dig into that a little bit on the next slide.
We also asked how often did students use closed captions when they were available, and we had 35% who responded that they often or always used closed captions when they’re available to them. It was important to us to know if students knew how to tell if a video had closed captioning as a feature and also if they knew how to turn on that feature. So a couple important numbers here is when we asked if they knew how to tell if it was there.
55.9% said yes, but we had another about 15% say no. And then, of course, about 30% said sometimes they know. And then if they know how to turn it on, the numbers stayed relatively steady. But another 15% said no, they don’t know how to turn it on. So I think it’s important to note that if we’re expecting students to have benefits from this, we may need to tell them how and when it’s there and where to use it, how to turn it on.
We also asked these same questions for transcripts. When students were asked how many videos in their courses had transcripts as an option, 43% noted that none are available. And for the same question that all, most, or many have transcripts, it was 11.2%, and that’s compared to 28.8% for closed captions. So less than half of that same number for closed captions.
We also found that they said, how often do you use them when they’re available, the never/seldom combined was a little over 63%. So quite a few students just aren’t using them. In terms of always/often using them, we had 19% when combined. That’s compared to 35% for closed captions. So we’re seeing transcripts being used less. But if you look at kind of how they’re being– this is kind of a slide that’s comparing those numbers– you’ll see we had almost the same response rates for those students, and never and seldom use is about double for transcripts than what it is for closed captions.
But when you compare their availability, you’ll see that there is almost four times the percentage of respondents that said no transcript was available than said no closed caption was available. And so what we’re seeing is that students are either more aware of the closed captions or transcripts are not being provided as much. Obviously, it’s hard to use a resource if it’s not available or if you don’t know it’s there. Importantly, about a quarter of respondents were not sure about closed caption availability, and about a fifth were not sure about transcripts. So they may not know where to look for them, but it sounds like they’re certainly not being announced or students aren’t paying attention to that announcement.
We also asked about the helpfulness that students experience from closed captions versus transcripts. And I wanted to combine this data to kind of show that we’re seeing pretty different numbers. For the closed caption question, we had about 1,500 students respond. For the transcript question, we had about 1,860 respond. So a little bit more responded for the transcript question.
So what you’re seeing on this slide is based on percentage of the whole for each of the questions that were calculated separately, and what you’ll see is that students, maybe 1 and 1/2%, said that they find closed captions not at all helpful. And the vast majority of them found it moderately, very, or extremely helpful to have closed captions. Whereas for transcripts, we had 31% say that they were not at all helpful, and then significantly smaller numbers for the moderately, very, and extremely.
So for closed captions, the highest percentage of responses said they were very helpful. For transcripts, the highest percentage said they were not at all helpful. And again, I would refer back to the idea that transcripts are not always there, which might be why some students are saying they’re not helpful. We did not collect that specific data. But we do have a little bit of data about hindrances that I’ll get to later on, and that may help explain a little bit about that.
We asked students why they used closed captions, and we found that the top three responses were that they help students focus, they help them retain the information, and they help overcome poor audio quality. You’ll also see that there was a response “Other, please describe.” Some of the kinds of things that we heard from students in that category were things like, “I live under a flight path, and planes are flipping loud,” and “ADHD makes audio processing hard, but captions help me to process the audio.” We also have pretty extensive qualitative data that I’ll talk a little bit about later about how students are using closed captions.
And in that qualitative data, we had a little over 1,200 comments from students that we included in our analysis, and five main themes emerged from that. And it was that students found closed captions helpful for environmental reasons, like they needed to study in quiet environments, because video/audio quality might be poor, or an instructor was difficult to understand. It was more convenient to use closed captions. They used them for accommodation purposes, or they used them as a learning aid.
And I wanted to share a few comments just for each of these so you can get a sense of what students were saying throughout this survey. So for environmental, we had a student say, “I don’t like playing videos with sound when I’m in an environment where doing so would be disruptive.” For the video and audio quality, students said things like, “It helps when a video has poor sound quality.” For the instructor being difficult to understand, we heard things like, “My professor has a slight accent that makes her words fast and jumbled.”
For the convenience category, students said things like, “I find it easier to read along with the videos rather than just sit there and listen to the professor talk.” And for accommodation, we heard from a range of students who are deaf or hard of hearing, but we also heard from a range of other disability categories, such as this one. “I’m dyslexic, so it helps me to know that the notes I’m writing down are both spelled correctly and in the right syntax.”
We also had a broad number of students who had kind of comments that were tied to specific learning benefits, and we broke those down further into these four categories of accuracy, comprehension, retention, and engagement, where students said things like, “If I need to write down notes, I know exactly how to quote a video.” For comprehension, “It can help me deeply understand the lecture.”
For retention, they said things like, “By simultaneously reading and listening to the content, I am able to retain the information better.” And for engagement, they mentioned how “They help me to focus on the video instead of just tuning out the noise.” And more information about each of these categories, including kind of breaking down how many comments fell into each one, is certainly going to be included in our larger study report that we’ll be sharing out later this fall.
In terms of some of the perceived hindrances of closed captions, almost 90% of respondents said that closed captions are not at all or only slightly a hindrance. So I think that’s an important thing to note. We also asked them to specifically respond qualitatively to what were the hindrances that they found with closed captions, and this we have done a little bit of the kind of counting on.
We know that 41.5% said that closed captions were distracting or required too much of their cognitive load. 34.8% said that closed captions included incorrect information, such as typos, or were incorrectly synced with the video. And 32% said that closed captions blocked important information. Those were the three highest categories of what came up for hindrances out of 400 qualitative comments that we analyzed.
For each of those areas, just to give you a sense of some of the kinds of things that students said, for closed captioning being distracting or requiring too much cognitive load, students said things like, “I do not focus on the video, I focus on the reading,” “If the video is simple with simple vocabulary, then closed captioning can be annoying and seem repetitive, causing the feeling of boredom,” and “They are distracting from the actual video.”
For comments about including incorrect information, such as typos or being incorrectly synced with the video, we had students say things like, “Most of the time, the videos in class have used automatic captions, which never match up with what is really being said, which often makes them inaccurate and not helpful.” Another student said, “When captioning is wrong, it is frustrating, because in a PhD program, details matter. Additionally, when they are wrong, it takes that much longer to process and contribute to a conversation.”
For the third category of closed captions blocking important information, students said things like, “Some instances, it covered up the equations needed to see for the math class,” “It can block important images or pictures,” “When they are in the way of the video, specifically words such as names or positions of people talking in the video.” So that gives you a little bit of a sense of how students are using closed captions for their learning and in various uses for their courses, but also some of the things that they’re perceiving as hindrances that are coming along with those closed captions.
I’m now going to shift to talking about why students are using transcripts and some of the hindrances that they’re finding with transcripts. So when we asked students why they used transcripts, the three top categories were that they used them as study guides, it helps them to retain information, and it helps them to find what they’re looking for. We also have an “other” category that had a pretty significant amount of responses.
Of the 262 responses we had for “other,” we coded 90 of them and found that students said things like, “I live in a rural area, and internet service is not available. Transcripts allow me to study lessons without accessing the video replay.” So in some cases, the transcripts were really tied to access. Another student said, “The video wouldn’t play. I read the transcript instead.” So students were using transcripts in some cases because they did not have access to the video.
We also talked about in our qualitative dividing into five main themes for the perceived benefits of transcripts, and you’ll see that these themes are very similar to what we found for captions. Perhaps that’s not too surprising. But some of the ways that students described these benefits are slightly different.
So for example, convenience is still a theme. But we had a lot of students talk about searching for information in the transcripts, and that was not as big of a theme with the closed captions. We also saw students say a lot about how they were using the transcripts for citations and quoting and copying and pasting from transcripts into their notes and things like that, and that was also not something that was occurring with the closed captions. So we see kind of slightly different uses of each of the mediums, but we’re also seeing them fall into generally the same categories of benefits.
For these qualitative comments, we analyzed 831. And broken out, you’ll see that they said a range of things. For environmental, one student said, “They helped me understand what was being said if I was in a noisy environment or when I was in a quiet environment. I found it easier to follow along as well.” Video and audio quality, they said things like, “They allow you to read what is said without having to understand the speaker clearly.”
For an instructor who is difficult to understand, a student said, “It is not always easy to understand professors who have a foreign dialect or who speak too quickly.” We saw accents mentioned a pretty significant amount of time in both the closed caption and the transcript comments.
For convenience, a student said, “It’s much easier to locate the key ideas and terms.” One of my favorite qualitative comments from this was someone who just wrote “Control-F” for how they saw a benefit of transcripts. So they can find information easily when they are searching through a transcript. And for accommodation, one student said, “It helps with my learning disability, being able to read the information while watching the video.”
We also broke out the transcript comments into specific areas that they’re being perceived as a learning benefit for the students. So in accuracy, one student said, “If I missed anything, I could have it right in front of me.” For comprehension, one student said, “It is harder to take notes from a video. Transcripts allow easier identification of key points.” For retention, a student said, “I am a visual learner, so being able to read the material at my own pace and take notes helped me to retain the information better.” And for engagement, a student said, “It helps me not to have to take notes, which allows me to focus on the video.”
We also asked students about their perceived hindrances of transcripts. And similar to closed captions, a little over 90% said that they were not at all or only slightly a hindrance. So students are perceiving both closed captions and transcripts as being significantly more beneficial to them than they are seeing them as a hindrance, which I think is a very important point.
We analyzed 153 qualitative comments for this particular area of the study and the survey responses. And we had 43.8% say that the transcripts were distracting from the video or visual cues or required too much attention or cognitive load. So very similar to what we saw with some of the responses for the closed captioning hindrances.
20.26% said that the transcripts included incorrect information, such as typos, were not well written, or were not formatted well. A couple students– well, more than a couple– commented on how transcripts can sometimes be just really big blocks of text that aren’t given paragraph form, and that was a challenge for them. And 14.38% said that transcripts were too long or too much to read or required too much time. And these were the top three categories of the perceived hindrances of transcripts. It’s pretty important to note that the top couple categories that overlap for transcripts and closed captioning are really tied to things that could potentially be fixed through quality assurance in terms of blocking information and typos and syncing problems and things like that. So that’s kind of interesting.
We also had a couple other students– well, I’m going to go, actually, ahead and break down these three areas together so you can see comments from each one. So for students who said the transcripts were distracting from the video or visual cues or that they required too much cognitive load, students said things like, “Transcripts are meant to be read without the video playing, so you cannot see an object if the person is pointing to it in the video.” They also said, “It’s hard to read and listen and watch at the same time.”
For students who commented on things like typos or transcripts not being well written or formatted well, some said, “Contained info that was irrelevant.” One said, “Not all correctly transcribed and often have weird line breaks,” and “Hard to sift through big blocks of text sometimes.”
And for those who commented on transcripts being too long, too much to read, or requiring too much time, they said, “It can make studying take longer than it should,” “It’s time consuming when you have to request for transcripts,” and “I would have to read it and not be able to watch the video and might fall behind with the other classmates.”
There were several students that also commented on various kind of other hindrances that came up for them. Some of the things that they said was finding a printer that doesn’t cost money to use poses to be difficult at times. Several students did comment on the inconvenience of having to find printers to use transcripts, so that was interesting.
There were also a couple students who commented specifically on having a preference for closed captions, and that was what they thought was a hindrance about transcripts. It’s just that they weren’t closed captions. So they said, “They’re not always really easy to find. I wish it was more like closed captioning and that there was just one click somewhere to turn it on and off. Some transcripts, you really have to search for.” We also had several students comment about how it was just difficult to find them within a learning management system, and so they had a hard time locating where they were.
So I’d like to talk about some possible takeaways just thus far for the data that we’ve analyzed for this particular study on student uses and perceptions of closed captions. One takeaway is that almost 100% of the students who responded to the study said that their courses include some video content. And so clearly students are engaging significantly with video in both their face-to-face and their online environments.
Generally, we found that students find closed captions to be more helpful than transcripts, but neither are considered to be much of a hindrance. And also, it should be noted that transcripts were more difficult for students to find. So all of those factors combined may impact why they find one to be more helpful over the other.
It’s also important to note that in the study, we did not ask specifically about interactive transcripts. We asked specifically about transcripts that can be printed out and used. And so it’s possible that interactive transcripts may have a different use or perception from students.
We know from the study that closed captions or transcripts are not being provided for all videos and multimedia items or that students are not aware when closed captions or transcripts have been made available. And so that’s something to consider in terms of just working with instructors and helping students to know where they can find these things and how to turn them on and making sure that’s very clear.
We also see from the study that closed captions and transcripts are being used in a range of ways, and in particular to aid student learning in a range of ways, and that students are using them for things like study aids. And they’re using them to make their learning more efficient. And there were lots of different kinds of creative uses that students were finding for transcripts and for closed captions. A lot of the details of that are going to be outlined in further detail in the study report.
We also found that many of the common hindrances that students noted might be mitigated by instituting a quality assurance process that particularly involves the creator of the content. So concerns about things like typos or blocking important information, those things might be able to be fixed relatively easily, depending on who’s reviewing the video and making changes to it as needed.
So I’m going to go ahead and pause to see if there’s any questions. And I do have some more information that I can provide about the study reports and an upcoming webinar, as well. But I’d like to go ahead and pause here to see if anyone has questions about the data that I’ve presented thus far or about the study overall.
LILY BOND: Thanks, Katie. There are a lot of questions coming in, and I want to encourage people to continue to ask those as we get into questions. Someone was asking, “Did you find a correlation between the 70% of students who had difficulty paying attention in class and the use of closed captioning?”
KATIE LINDER: That is an excellent question. So that is actually the next part of our data analysis. We’re going to be doing some subgroup analyzes of really drilling down to see for that population in particular, but we’re also going to be looking at students who identify English as a second language. We’ll be breaking out the different disability status of the students to see if there are higher amounts of students who are engaging in closed caption use who may view it differently in terms of benefits and hindrances. So all of that is going to be coming up.
We are considering doing a third. We have our second webinar coming up, but maybe a third webinar that really focuses specifically on that subgroup analyzes. So if that’s something you’re interested in, know that it will be coming in the report. But there will also probably be a webinar about it also in the future, so keep an eye out for that.
LILY BOND: Thanks, Katie. And several people are asking whether or not this study is being published. It is, and everyone who registered for this webinar will be notified when the study is released.
KATIE LINDER: Yeah, we’re creating a report right now, and I can just give a little bit of information about that. The report will include additional results from what I talked about today, including more specific qualitative analyzes. The survey instrument will also be included in that study report, and also data tables for all the questions. So you will be emailed when that’s available. A second report is also being created for the institutional study.
I do want to mention right now, because so much of the information is being included in the study report, I don’t have plans to publish it out as peer reviewed, if that was part of that question. I’m not sure. But that’s something that we may consider in the future, depending on what we find in some of the subgroup analyzes and the other things that we’re looking at right now.
LILY BOND: Thanks, Katie. There are a couple of questions about the caption quality. “It looks like a lot of the people were experiencing auto-generated captions. Do you have any idea of what percentage of the captions were auto-generated captions, or what percentage were inaccurate versus how many were accurate caption files?”
KATIE LINDER: So that’s a really, really great question. We did not ask students to specifically talk about that. We may get to a little bit more of that information in the institutional study, because we do ask about how are they captioning things and what are kind of the methods that they’re using. That is a smaller study. It didn’t have as many respondents, obviously, as this one.
But I think that we– one of the things that we can kind of pull out of the quantitative responses is how many students focused on that specifically. Having just kind of been coding this data over the past couple of weeks, I can tell you that it wasn’t something that was mentioned frequently. I mean, it certainly came up, but it wasn’t an overwhelming theme where students were saying this was auto generated. I think the bigger issue for them for closed captioning was really blocking of information, and also maybe not syncing correctly and things like that.
But, yeah, it’s something that we did not ask specifically about. And so maybe something will come out of that institutional study that will be pertinent to that. But for the student study, we don’t have anything definitive.
LILY BOND: Thanks, Katie. Another question here is, “Did you define what closed captions were to the participants?”
KATIE LINDER: Yes, we did. So in the survey instrument, before we started the questions on closed captions, we included a definition. And that definition was kind of collaboratively created by our Research Unit and with 3Play Media. We also had a disability studies administrator and kind of an expert in disability studies look over the survey as well just to make sure that we were wording things correctly, particularly when we were asking students about particular disability categories and things like that. So they had a chance to review that definition.
And then before the transcript questions, we also included a definition of transcripts that clearly noted that we weren’t talking about interactive but something that you could access and print out to read later. And so several students, a big theme that came up, particularly for the transcripts, was they like to go back and review it at a later time, and that was something that they spent a lot of time with in terms of studying and notetaking. So for both of those categories, we did provide a definition before we entered into that grouping of questions.
LILY BOND: Thanks, Katie. Someone else is asking, referring to slide 41, “Were students asked about the typical length of the video from which the transcript was derived?”
KATIE LINDER: We did not ask students about typical length of the videos. That was not something that we included in the questions. That’s a great question, though, and an excellent point. It’s important to think about, especially for the concern that students were saying these were too long. Any time you have a transcript for a 60-minute video, I think we might all agree it’s too long. So, no, we have no sense of that context for this particular question.
LILY BOND: Thanks. Someone else is asking, “Were you surprised that so many students liked captions, even when they weren’t in accommodation? One of the arguments movie theaters use for not open captioning films is that the captions are distracting, but maybe not.”
KATIE LINDER: I love that question. So I actually was not surprised, just because of some other work that we’ve been doing and because of the pilot study we did here at Ecampus that found students were using them very broadly, and particularly for things like a child is sleeping in the same room and they can’t watch the video unless they turn on the captions. Several people in this study also mentioned that they use captions all the time, not just for their learning experience. Captions with Netflix was mentioned multiple times. And so we know that students are engaging with them outside of the classroom.
I do think, though, one of the reasons certainly for me to do this study is I do think that institutions need rationales for why they should be more broadly captioning. And so I was certainly hopeful that we would find a broad use of captions and certainly a use of captions for learning aids, and that was something that absolutely came out of the study. So I think that it’s really important for us to be thinking about letting students who do not request captions or transcripts as accommodation, letting them know when those things are there, and potentially even how they could possibly be using them. We have some students who are creative, and they’re thinking about how to do that, and they’ve already incorporated this into their learning experience.
But as we saw from earlier numbers, clearly there are students who are not and aren’t aware that these are even there. Several students commented that it’s just never been available to them. Many students in the qualitative comments said things like, it’s never been available to me. But I imagine if it was, it would be really helpful for X, Y, or Z. So there were students who just had never been given the opportunity to use these things.
LILY BOND: Thanks, Katie. Another question said, “Do you know how some of the institutions sent out notifications to students about the survey?”
KATIE LINDER: We know that institutions were given– so when institutions signed on to recruit for us, we sent them IRB-approved recruitment materials. And so there were two ways that they could communicate with students. The first was via email, and it was an email that was targeted specifically to students for each institution saying that your institution has partnered with Oregon State and 3Play Media. The second was we offered a smaller kind of short paragraph blurb that could be included if there was a newsletter email going out or some other kind of broader mechanism where they were advertising to students. They could not send a full email, they could send a short blurb, and that was also IRB approved.
And so after the initial recruitment messaging was sent out, institutions were then reminded twice to send it out again. And so I was in contact with institutions during this entire kind of data collection period and then making sure that students received that messaging up to three times. So we had primarily email recruitment, and then potentially some people may have also used that little blurb in other– primarily online, I would imagine– publications that they sent out to students.
LILY BOND: Thanks, Katie. There are several questions about the correlation between having trouble finding captions and the institution. Specifically, people are wondering if it’s a problem tied to a specific type of video player or if you’ve found that the factors that affected students having a negative impression could be resolved with a universally designed video player.
KATIE LINDER: So I can’t offer information from the study on that. We don’t have the information and the data. But I can offer an opinion on that, which is just I think that, certainly, if you have a player where it is unclear how students can access the captions, they are going to be less likely to stumble upon it and use it, particularly for a learning aid.
But also, one of the things we found– and it came out more in the Ecampus study that we did here at Oregon State than I’ve seen yet in the national study– but that students like to toggle closed captions on and off. They don’t always turn them on for the entire video. And so we heard a lot of people say, I like to turn them on if there was a word I didn’t know or if I didn’t understand a short part of the video and what they were saying.
So that is another piece, is that students are not sometimes just turning them on for the entire video and utilizing them that way. Sometimes, they like to turn them on in the middle of things. And so that’s actually impacted how we designed some of our multimedia here at Ecampus and making sure that there’s a closed caption button on each page so that students can turn it on throughout and it’s not just offered to them at the beginning of a multimedia or a video experience.
Now, I don’t know enough about all the players to know how they all work in that way. But I do know that, certainly, and specifically with multimedia, sometimes it’s not something that’s included on every single slide or every single page that a student might be engaging with. So I don’t have data from the study that I can share, but I think it’s a really interesting question and one to dig into further.
LILY BOND: Thanks, Katie. You were starting to mention this, but someone is asking if there are any key actions that Oregon State will consider taking based on the results of the study, like planning for a closed captioning budget.
KATIE LINDER: Yeah. So in Ecampus specifically– which is really all I can speak to for this, because how we’re sharing this out within Ecampus is different than how it might be impacting our campus wide. And I know our campus more largely is working with vendors in a range of ways– but with Ecampus in particular, this has already impacted our practices and helped our multimedia designers and our instructional designers just have a better idea of how students are engaging with captions. I think we had previously a lot of anecdotal evidence about how we thought maybe students were engaging, and this really gives us, I think, a bigger picture of the range of ways.
At Ecampus in particular, we have a very large adult learner population in our online courses, and so we got a lot of responses about things like, I’m juggling children and all these other kinds of factors where I wouldn’t be able to watch the video save for having closed captions. And so it really does become, for many of our students, an access issue beyond accommodation. And that’s something that I think has been really important for us to learn. But this is also something that we’re just kind of getting into, analyzing the data.
Certainly, we will be sharing it out on our campus, but also more broadly through this report. And I’m sure, I hope, it will generate some additional conversations here about how we can use the data to move forward. Certainly, having data like this is very helpful to help us understand our student populations and how they’re using these kinds of tools.
LILY BOND: Thanks, Katie. Another question here is, “I never considered the benefits of closed captions for use of helping to understand instructors that have a heavy accent. Are the instructors with this issue using captions more than other instructors where that is not an issue?”
KATIE LINDER: I think that’s a really great question, and I don’t know the answer to that. It’s not something that we asked. And actually, even our institutional survey was really focused more toward administrators and not to faculty. So I think that’s a really interesting question and might be kind of institution specific, depending on the amount of instructors who have accents that are teaching for the institution.
The other thing we did hear, though, and actually very frequently, was that students use closed captioning quite a bit when they’re taking language courses, because they like to see the foreign language printed out while they’re hearing it. Because they’re not used to seeing it and what it would look like, and they feel that it helps them learn the language better.
So we saw it for the accents. That was reported quite a bit in terms of helping to understand faculty with accents. But on the other side of things, we also really saw it for being beneficial in foreign language courses. Students definitely differentiated between– they weren’t using it to get English subtitles. They were using it to see the foreign language in print so that they could understand the connection between what they were hearing and what they were seeing.
LILY BOND: Thanks, Katie. Someone else is asking, “Did the research ask about the use of captions outside of the classroom? Like if captions on videos that are used for advertising on campus are captioned and whether or not that helps, or if students showed the same amount of interest in videos inside and outside the classroom.”
KATIE LINDER: That is a great question. The student study did not ask that. We actually focused– and I should clarify, too– specifically because the survey was getting quite long and we were really trying to make sure we were getting really good data about closed captions and transcripts, we had to get really tight with our focus. So there’s so much more that could be asked about this.
So in the student study, we did not talk about those things. In the institutional study that we did– and we’re going to, again, be presenting that in a webinar on October 12– we got into more about how institutions are differentiating between face-to-face video, online video, and video that’s created for institutional purposes and if those are kind of prioritized for captioning or if there’s a different process for captioning those things based on the purpose. So we get into a little bit more of that in the institutional study. It’s certainly not getting to exactly the question you’re asking. But I think your question really points to the fact that there’s quite a bit more here that we need to be asking about closed caption use and transcript and interactive transcript use.
LILY BOND: Thanks, Katie. I think that you’ll probably get into this much more in a student subgroup webinar, but several people have asked if you noticed a difference between people who used captions as native English speakers versus people who know English as a second language.
KATIE LINDER: That is something we will definitely be digging into. I don’t have that in front of me right now. We’re going to be including that for sure in the webinar that we do on the student subgroup analysis, and there will be a breakout of that in the study report as well. So that’s definitely kind of our phase two of what we’re looking at here, and I think it’s really important. There’s actually several subgroups that we included in the survey so we could really drill down and see what was going on with each of those populations.
And I can tell you, just from initially what we found in our Ecampus study that’s just Oregon State specific, there were differences between how students in the sub-populations were kind of talking about their use of closed captions and transcripts. We saw certain subgroup populations that saw them as being no hindrance whatsoever. I mean, they just didn’t comment on hindrances at all and only framed it in terms of benefit. Which is not surprising, given different populations like English as a second language. So that’s coming. and I think those questions are really key to this study, so we’ll be excited to share out more of that at the third webinar that we put together.
LILY BOND: Thanks, Katie. A couple of people have asked whether or not you differentiated between captions and subtitles or closed and open captions.
KATIE LINDER: We did not. In our definition of closed captions– which, unfortunately, I don’t have right in front of me. It will be part of the survey instrument that’s included in the study report, so you can see exactly what we said to students. We talked about kind of just the definition of closed captions.
Given kind of looking through the coding, it’s pretty clear that students were not seeing it as subtitles. And even the ones I can kind of pull out– and we may kind of code specifically the people who talked about foreign language– it’s clear that they’re talking about words that are in the same language as what’s being spoken. For the open versus closed, that was not something that we asked about or that we defined in the study more specifically, so that’s not included.
LILY BOND: Thanks, Katie. It looks like the questions are slowing down. If anyone has other questions, feel free to ask them. But Katie, you might want to wrap up.
KATIE LINDER: Great. All right, so I did want to make sure everybody knows about the upcoming webinar we’re going to do on October 12. And we can go ahead and put the link in the chat window, so if you’re interested in registering for that. This is going to be more specifically on how colleges nationwide are handling captioning, and we’re going to talk about kind of how institutions are prioritizing captioning. How are they budgeting for it? What are the kinds of captioning solutions that they’re deciding on? Are they using kind of internal solutions or extremal solutions? And a bunch of other data that came out of this study that was specifically to institutions. So I’m really excited to share all of that with you guys.
The other thing I wanted to mention was another resource that might be interesting to you if you’re interested in kind of the research side of what we did with the study. Our Research Unit here at Ecampus also produces a weekly podcast about topics and issues related to research in higher education. And full transcripts are provided with each episode, so it is an accessible resource.
It’s also a resource that comes with instructor guides. We’re hoping that it will be used in the classroom as well with students. So I think I’ve got a link that we can post, too, in the chat window. I think Lily has a link for that as well. So if you’re interested in learning more about that, you can.
And also, I just want to thank all of you for your time and attention. Clearly, this is a topic that is of much interest to all of us. We always want to improve student learning as much as we can. And I encourage you to contact me if you have additional questions about the study, if you have questions about other work that we’re doing here at Oregon State University Ecampus with our research. And of course, I would love to connect with you on Twitter. So thank you so much for your time and attention. It’s been really wonderful to hear your questions and get a sense of what your interest is in this research.
LILY BOND: Thank you so much, Katie. That was a really great presentation, and everyone is so excited about all of the results that you’re able to share and eager for more webinars and for the full study. So we will keep everyone posted on those. And I hope everyone has a great rest of the day. Thanks, everyone.