Posted: 11/25/2013 9:27 am
Or is it?
In order to answer that question, we need take a look at the calculation used to determine these alleged drop-out percentages. It's actually not all that complex since it starts by taking the number of people who hit the enroll button on a site such as Coursera or edX and sticking it into the denominator of a fraction. After that, it's just a matter of placing the number of students who earned a certificate of completion into the numerator and, voila, you end up with your completion rate!
The completion rate is actually the opposite of attrition, but for purposes of this discussion we'll use it to see what happens to the percentage of course finishers if we start looking more closely at the numbers used in our fraction (particularly the one below the line).
Basing our denominator on enrollments assumes that everyone who hits the enroll button on a MOOC web page should be considered the equivalent of a college student who signs up to take a course at their university. But is that an appropriate assumption? After all, many schools that allow shop around periods understandably choose not to include shoppers who don't end up taking the course in their final drop-out calculations. Similarly, schools that permit auditing treat auditors differently than standard enrollees when calculating things like course completion rates and averages grades.
Given that there is no punishment for clicking on that Enroll button on a MOOC site, even if you never had any intention of taking the class to completion, it's appropriate to ask whether "enrolling" in a MOOC is equivalent to enrolling in a traditional college course, shopping around, browsing through a college catalog or simply filling out an online form that allows you to get something for nothing.
Jeffrey Pomerantz from the University of North Carolina at Chapel Hill offered a Coursera MOOC entitled Metadata: Organizing and Discovering Information in 2013 and recently summarized findings derived from analyzing course data here. And one of his more interesting analyses involved seeing what would happen if you used different methods for defining who actually signed up for a class with the intention to complete it (vs. audit it or just check it out).
Using the number Coursera sent him of "Total Registered Students" (i.e., the number of people who hit the Enroll button) as a denominator does indeed give you a completion percentage of 5 percent. And if you instead use Total Active Students (the number of students who logged onto the site at least once after registering) that completion rate climbs to 10 percent (still within the range MOOC critics use when they complain about attrition).
But if you use the number of students who watched at least one video as your denominator, completion percentages climb to 15 percent. And if you make the assumption that only students who complete at least one assignment (even a short quiz at the end of lesson 1) should be considered serious enrollees, his completion rate skyrockets to 48 percent.
This last figure jibes with one of edX President Anant Agarwal's key talking points when confronted with questions regarding MOOC attrition rates in which he claims pass rates climb above 40 percent when you assume that only students who actually finish an assignment are demonstrating a desire to take the course to completion. And if you take into account that many of the students who might enroll in a course but not do any assignments might be auditing the class by watching some or all of the lecture videos, then you begin to get a better picture of what people are actually doing when they take part in a MOOC (other than dropping out).
Keep in mind that this better informed picture is a two-edged sword for MOOC boosters. On the one hand, it gives them strong evidence to counter critics who like to hammer on drop-out rates to "prove" MOOCs are educationally worthless. At the same time, it's hard to continue using huge front-end MOOC enrollments to impress the media, public and investors and then turn around and minimize the significance of that number when it comes time to calculate more meaningful completion statistics.
And while it's certainly true that four figure graduation rates are less sexy than the six figure enrollment numbers thrown around during the period of MOOC over exuberance, we need to keep in mind one more thing the numbers are telling us, best summarized in the last word which I give to Professor Pomerantz:
Before my MOOC launched, I did a quick back-of-the-envelope calculation of how many students I've ever had in the classroom, since I started teaching in grad school. And the number I came up with was, approximately 1400. The number of students who completed my MOOC is approximately equal to the number of students I've had in the classroom in my entire career. The number of students who were active in the MOOC (Total Active Students) turned out to be approximately an order of magnitude more than the number of students I've had in the classroom in my entire career. Contemplate that.
