Jack's Post Piece on the Data We Need.

Jack's new Washington Post piece raises the question of The measures we really wanted, which would have helped us piece together a more accurate picture school quality—measures of teacher job satisfaction, student happiness, parental engagement,

These are things that we can easily measure. 

Somerville was lauded for its use of the city census to measure citizen happiness.  District surveys of students such as the Yotth Risk Behavior Survey provide an instrument to access this information.  It makes good sense to measure this information.

g.

print

---

The school data problem: what we have vs. what we need

By Valerie Strauss, Updated: November 5 at 2:00 pm

(By Alan Brandt – AP)

It’s the backbone of the modern school reform: data. And a lot of it, it turns out, isn’t any good. Here’s a post about the problem of the data we have vs. the data we need, by Jack Schneider, an assistant professor of education at the College of the Holy Cross, who confronted the issue recently when he helped the Boston Globe create a rating tool for schools in Massachusetts. Schneider is the author of “Excellence For All: How a New Breed of Reformers Is Transforming America’s Public Schools” and the upcoming ”From the Ivory Tower to the Schoolhouse: How Scholarship Becomes Common Knowledge in Education.” He tweets @Edu_Historian.

By Jack Schneider
For the past two years, Secretary of Education Arne Duncan has been striking bargains with state departments of education, waiving some of the most unreasonable and restrictive accountability provisions of federal education law. Desperate for greater flexibility, the vast majority of states ultimately hammered out deals with him. And in the process, they gained greater flexibility in the use of federal funds, in creating new performance indexes, and in outlining more attainable aims. Yet as educators in those 43 states are beginning to find out, one key component of life in the age of No Child Left Behind—the dominance of standardized tests—isn’t going anywhere. Tests still reign supreme.

Congress’s 2002 reauthorization of the Elementary and Secondary Education Act, commonly known as No Child Left Behind, gave states 12 years to produce universal proficiency as measured by standardized test scores. And with 2014 just around the corner, none have come anywhere close. Without waivers from the law, each would lose its share of federal aid for low-income students—an outcome that both Democrats and Republicans see as totally unacceptable. Yet a faint-hearted Congress has continually failed to act. So, out of necessity, Duncan has stepped in to stave off the doomsday scenario. But he has also taken it upon himself in this waiver process to decide what better accountability measures might look like, insisting on continued heavy reliance on standardized test scores despite all that we’ve learned in the past decade.
Standardized tests given to K-12 students are not without merit. They can function as clear indicators of basic academic competencies. And they can play an important role as diagnostic tools. But they capture only a fraction of life in schools. Built almost exclusively around multiple-choice questions, such tests tell us nothing about a student’s ability to think or write or persuade, to perform experiments or conduct research, to paint, or to play an instrument. They provide no insight into a school’s social climate, its academic orientation, or its general culture. And, as any teacher can explain, the testing and accountability movement has also been plagued by a number of unintended consequences. The school curriculum has narrowed. Test-prep now takes up an inordinate amount of instructional time. And teacher autonomy has withered.
It is clear why Arne Duncan and like-minded reformers favor standardized tests. Along with high-stakes accountability mechanisms, such tests have given policy leaders at the state and federal levels an unprecedented ability to pry open the classroom and control instructional delivery. Equally important to them, standardized tests have yielded a cascade of data that policy elites have assembled into a picture of school quality—constructing evaluative report cards, and even tying student achievement scores to particular teachers as a means of calculating “value” added.
To their credit, policy analysts, continue to get better at slicing this narrow bandwidth of data. Student growth measures, for instance, are innovative instruments that produce a level of comparability across dramatically different populations. Yet no matter how creative they get, analysts will never be able to assemble a full picture as long as they are working with a single piece of the puzzle. And currently, that’s all they’ve got.
Recently, a colleague and I—along with The Boston Globe—built a school rating tool for the state of Massachusetts. Striving to go beyond raw test scores, we included measures like diversity, college matriculation and school resources. We also included student growth on state tests, and drew upon figures like SAT writing scores and Advanced Placement results. And we think our tool does quite a bit to move the conversation forward around the question of school quality. Yet we were severely constrained by available data, even in Massachusetts where a relatively robust set of figures is collected and reported. The measures we really wanted, which would have helped us piece together a more accurate picture school quality—measures of teacher job satisfaction, student happiness, parental engagement, richness of art and music programming, and employee retention rates, to name just a few—simply are not collected in any systematic way at the state or federal level. We were working with our hands tied.
Our experience was instructive. If we were unable to find the data we felt that we needed to assemble a truly robust picture of school quality, it is hard to believe that state and federal policymakers working with the same data can do any better. And yet policy leaders seem genuinely untroubled by this. The waiver process, which should have been a time to reconsider the process of measurement—reflecting on what we value, what schools can really be held accountable for, and what the limits of assessment are—has instead been an opportunity to usher in more of the same. In Massachusetts, for instance, only real change to state accountability calculations is the inclusion of graduation and dropout rates; and standardized test scores will still constitute the bulk of any measure of educational achievement.
Many parents, educators, and scholars across the United States are concerned with the impact of standardized testing on K-12 education. But the undemocratic NCLB waiver process has precluded debate and thwarted dissent. Elevating the voices of a few, it has fended off deliberation and closed off opportunities for change. It has fostered the misguided belief that the data we have is the data we need.

© The Washington Post Company