Data is a big deal

By John Kuhn

Data is a big deal. A great deal of innovation is happening right now in the field of data collection, storage, and management in the field of education. There are some well-documented fears among parents and teachers regarding these trends. Who will control the data? How will the data be used? Will my child’s data be protected? The worst-case scenarios–data misuse, hacking, data misrepresentation, a great sabotage of American schools–are downright terrifying.

Some education watchers and commenters, meanwhile, are enthusiastic supporters of a more data-informed future in our schools. They note the promise of more effective daily practice informed by accurate and timely information about student performance–more data than ever before is available, as are technological tools far more powerful than ever before, placed in the hands of educators. The best-case scenarios–individualized highly-effective remediation, personalized educational experiences, de-tracking and de-grading students, a great flourishing in American schools–are dizzying in their hopeful promise.

There is a great deal of conversation that occurs online about the various particulars surrounding educational data. Like much educational discourse in social media, it is fascinating, if occasionally (or often) strident.

It goes without saying that data is not inherently a bad thing. In fact, human history is decorated with moments wherein men and women made astounding discoveries on the basis of their analysis of extant data. From Madame Curie to Einstein to George Washington Carver–and outside of science too, in worlds as diverse as music and gastronomy and exploration–the greats have always relied on information they had access to when making their discoveries. Data is the raw material for effective decision-making. If you know there is an iceberg ahead soon enough, the Titanic doesn’t sink.

Data should indeed inform decisions. And, indeed, educators need data at their fingertips.

But data should come with some serious warnings, and we’ve seen just how bad things can get in education. The way I see it, there are at least 4 major dangers when it comes to data.

1. While data (plural) should inform, each datum wants to rule alone. In an America that is uncomfortable with nuance, we have two dominant political parties, two dominant soda brands, and so on. We like to reduce things to manageable-if-extremely-imprecise chunks. As such, a single datum such as a Standardized Test Score–like the ring in The Lord of the Rings–invariably wants to take over. Data-informed quickly gives way to data-driven, and then data-driven gives way to datum-blinded. And that is, in my opinion at least, where we live today.

2. Data wants all your time and money and effort. There is a dictum that says something along the lines that the more a certain measure counts in social sciences, the more likely it is to pervert the whole process of measurement. If test scores are everything and have “high stakes,” then it is practically inevitable that end users will short-circuit the system in a single-minded effort to get good test scores. In a similar vein, I propose that the more a certain datum counts, the more time, money, and energy–all of which are limited–will be devoted to it. If the test suddenly becomes THE MEASURE, we will at length convince ourselves that it isn’t enough to test a sampling of students. We must test them all. And we will convince ourselves that it isn’t enough to test them every few years. We must test them every year, and not only that, we must test them at the beginning, middle, and end of the year, in order to see progress on the one datum that matters. Because this data, it’s the thing. It’s THE thing. Education was THE thing, but now this one tiny datum is. Data is ultimately a great and tricky usurper. We are like a hunter who once hunted deer but then got sidetracked by obsessively examining deer tracks. We became experts at deer tracks. Now we hunt deer tracks. We make molds of them. We hang them on our walls. We haven’t seen a deer in ages, and we can’t really figure out why we’re so hungry. But we have a great spreadsheet that sorts our deer track collection by circumference, regularity, and a hundred other criteria. Because deer tracks are important for finding the deer, only we kind of forgot about the deer.

3. Data is useful for correcting course, but it is also useful for charting a course straight for the iceberg. Data, like fire and shotguns, is neither intrinsically good nor bad. In fact, like fire and shotguns, it can be a life-saver when used properly in the right circumstances, and it can be deadly when used improperly in the wrong circumstances. Teachers and parents who get labeled “anti-testing” (because, again, nuance is hard) are often not at all against testing. The vast majority of the so-called “anti-testing” teachers give tests in their classrooms. So it isn’t the test that motivates much of the opposition to reform. And it isn’t the data, either. It’s the fact that many, many stakeholders don’t trust the people hoovering up the data to use–they presume, because of their experience with the school reform movement as it has unfolded–against students, teachers, and schools. The vocal opposition we see to data collection efforts like inBloom, to curriculum standards (which define the data to be collected) like the Common Core, and to tests (the data source) like the MAP can all be traced back, largely, to two things: (1) dismay over how much class time is sacrificed for the all-encompassing data hunt, and (2) a foundational mistrust regarding the aims of those who gather and control the data. If your dad brings home a new baseball bat, it’s a pretty happy time in the family–unless your dad has been in the habit of beating the family with blunt objects. Data is that baseball bat. A better analogy might be a doctor who causes his patients pain unnecessarily with his medical equipment. Patients are naturally going to resist going in for procedures that the doctor says are “good for them” if they know it will come with excessive pain. There is a vigorous campaign online and in the papers and political buildings to discredit opponents of school reform as just so many Chicken Littles “defending the status quo” and sticking their heads in the sand. A salient question, though, is this: has the sector-controlling school reform movement, going back to the dawn of No Child Left Behind, wielded data honestly, ethically, and constructively? If not, then yeah, there will be resistance. These people aren’t Chicken Littles. They’re Chickens Who Won’t Get in the Pot. The deeper the mistrust, the more vocal the resistance.

4. Big Data hates little data. Data has always been gathered by teachers, and it has always informed their instruction. Teachers give assignments and grade them–not because they like to grade, but because they want their students and themselves to see whether or not students are learning the material. But Big Data isn’t apparently interested in this arrangement, never mind that some studies have demonstrated that “high school GPA is ‘the best single predictor’” of college success. Big Data–and the people behind it–appear to dismiss the trustworthiness of the classroom teacher. Maybe because–for some Big Data adherents, anyway–there is no way to monetize teacher grading. For others, it’s probably driven by a general disdain for the quality of America’s teaching corps. The upshot is that each teacher needs a set of standards or presumably he or she will teach 180 days worth of lessons about coloring or dinosaurs. Defenders of the standards often exasperatedly say things like, “But teachers need to know what needs to be taught each year, and what has been taught the year before and will be taught the next year.” Instruction must be aligned. And this is true, but teachers are smart enough to know–because they’ve all seen this movie–that standards have trouble stopping at “informing instruction,” despite protestations to the contrary. The standards are often followed closely behind with canned lessons and scripted curricula. Teachers in many schools–particularly those that struggle with low test scores–will be instructed to “say this on this day.” For quality control purposes, one could assume. Many school leaders will resist this type of silliness of course, but many won’t. The data gathered at the local level, like teaching choices made there, is likely seen as suspect. Test scores are the holy grail, the mandatory “objective measure,” because locally-developed data is subjective and controlled by teachers. If our religion of reform is built on a foundation of mistrust regarding the efficacy and quality of our teachers, we must avoid attaching too much weight to the data they generate and handle. We have taken it on faith that they are self-interested and will twist the data to serve their own careerist purposes. For this reason, calls for multiple measures in making judgments about school quality tie reformers in knots–there are limited “objective measures” available that completely disempower local educators and keep their hands off the controls. To get a truly broad array of multiple measures informing our system of gauging student progress, we will have to let teachers’ data count.

More thoughts later on rules that should drive data so that data doesn’t drive us. Like Isaac Asimov’s famous Three Laws for Robotics, we need an overarching moral document to police the Wild West of Big Data, or else abuse and destructiveness will win the day.

Recent Posts
 ■ That Anti-union Billboard in Times Square: Consider the Timing
■ Learning to teach reading from Whitey McWhiterson and her twin sister
■ On the defense of @rweingarten by @dianeravitch
■ Does this school really care for the children?
■ Rethinking “Creative” in the Common Core Era: “Let’s not tell them what to write”


Top Posts & Pages
 ■Does this school really care for the children?
 ■Guided Reading in Kindergarten is one of the many dramatic outrages of #commoncore
 ■@johnkuhntx: The Tyranny of the Datum
 ■Joanne Yatvin: The Common Core Standards May Be Harmful to Children
 ■An Open Letter to New Teach for America Recruits
 ■On the defense of @rweingarten by @dianeravitch
 ■That Anti-union Billboard in Times Square: Consider the Timing
 ■Learning to teach reading from Whitey McWhiterson and her twin sister
 ■The Costs of Common Core?
 ■Rethinking "Creative" in the Common Core Era: "Let's not tell them what to write"



Recent Comments
 ■ Peggy Robertson on Guided Reading in Kindergarten is one of the many dramatic outrages of #commoncore
■ Dottie on Does this school really care for the children?
■ Jim Horn on That Anti-union Billboard in Times Square: Consider the Timing
■ Peggy Robertson on Learning to teach reading from Whitey McWhiterson and her twin sister