Jack Schneider in EdWeek today

Jack is a fellow Somerville dad and Ed School professor at Holly Cross.  I had lunch with him last week (at Amsterdam Falafel) and found him to tremendously insightful and constructive in his thinking about schools.  Below is a piece he had published in EdWeek today which I like very much.

 g.

Commentary

How to Measure School Quality

By Jack Schneider & Anil Nathan

How do you measure school quality?

A seasoned educator can get a feel for a school in a day, though it might take a year to determine its particular strengths and weaknesses. Yet parents do not have that luxury; they cannot spend long stretches of time visiting all of the schools they might send their children to. And even if they did, many might not be sure what to look for.

Given this challenge, parents tend to rely on rougher indicators of quality—word-of-mouth and standardized-test scores. Parents weigh these two factors together and then make enrollment decisions with a relative sense of confidence. But these mechanisms are highly problematic and deserve careful scrutiny.

The most obvious issue is one of inaccuracy. A school can have a good reputation or impressive test scores because it caters to a privileged population. But, as research indicates, these high-scoring schools often add less value than some of their competitors with lesser reputations or lower scores.

"We believe it is possible to give parents richer and more useful information about schools."

A second problem is the effect that these indicators, particularly raw test scores, have on the system as a whole. Because standardized-test scores offer a narrow and opaque measure of performance—a measure most useful for myopically ranking schools against each other—reliance on them has ensured a small number of winners and a large number of losers. This, in turn, has damaged the reputations of countless schools and intensified the alarmist narrative of educational decline.

Finally, these methods of gauging quality are troubling because they have promoted segregation. Schools with strong scores and reputations become objects of intense competition and ultimately propel property values upward and price out the working class. Consequently, even if schools in lower-income neighborhoods are good, working-class children remain isolated from their more privileged peers, gain less access to social capital, and enjoy fewer of the resources that privileged parents bring to bear on their children's schools. And for their part, privileged students are denied engagement with those different from themselves.

Of course, merely demonstrating the inadequacies of these measures is not enough. Parents are desperate for information. And, insofar as that is the case, they will invariably opt for unreliable intelligence over none at all.

But what if we had better, more accurate measures?

"We urge scholars and policymakers to build additional tools to help parents make more informed decisions about the schools that best suit their children."

Recently, we collaborated with The Boston Globe in designing a school-rating tool that we believe will move the conversation beyond reputation or raw test scores. Pulling from the various data available to us, we include measures that reflect something about school quality. And recognizing the fact that rating schools is an inherently subjective enterprise, our model allows parents to customize rankings based on personal values, including school culture, college readiness, and diversity.

The tool is imperfect, certainly. After all, such measures are constrained by available data, which is not necessarily what we might wish for in an ideal situation. And every state approaches data collection differently, so our model is not perfectly replicable. In Massachusetts, for instance, the state generates a student growth percentile, or SGP—a measure that gauges growth by comparing one student's history of exam scores with those of all the other students in the state with a similar testing history. On the whole, however, our tool turns the tables on some conventional thinking about school quality in the state, and we believe it will change the way parents make decisions.

So what are our measures?

The first two are drawn directly from the state using SGP for English/language arts and mathematics. Such information, we recognize, is not currently available in all states. But many are moving toward a model that does a better job of leveling the playing field between groups with different background characteristics.

Our third and fourth categories are "school climate" and "college readiness." For the former, we use graduation and dropout rates, proxies for student and adult commitment to the process of education, as well as high school seniors' college plans. For the latter, we use SAT writing scores and the percentage of students scoring 3 or higher on Advanced Placement tests, two relatively strong predictors of college success. Such data are widely available and, therefore, this aspect of measurement is easy to reproduce.

Lastly, we include "school resources" and "diversity." While research is mixed about the exact impact of money, it is clear that greater resources can afford a wider range of opportunities for students. And while not all parents value diversity in schooling, it is a factor that many consider invaluable in the process of education. Our measure for school resources (expenditures per student) is straightforward. But in order to measure diversity, we have to be more creative. We imagine a level of "perfect diversity" for schools and then calculate the distance between a school's actual population and that ideal.

Again, our tool is a limited one. Graduation from high school and intent to pursue higher education are hardly ideal proxies for school climate. Our measure of college readiness relies, imperfectly, on standardized-test scores. We have an imperfect picture of school resources, tracking how much money is spent but not how it is spent. And our diversity calculation might be criticized as somewhat arbitrary: It is based on a scenario in which white, African-American, Latino and Hispanic students, and students of other racial or ethnic groups each make up 25 percent of the school's population. We freely admit all of this, and we have encouraged our critics to join us in advocating for the systemic collection of richer and more varied data at the district or state levels.

But despite these limitations, we believe this is the first step toward creating a better source of information for the public. As such, we urge scholars and policymakers to build additional tools to help parents make more informed decisions about the schools that best suit their children.

Beyond providing better information to parents, however, our interest lies in restoring some breadth and sanity to the way we evaluate schools. Americans are bombarded by policy talk framing most schools, particularly those not located in leafy suburbs, as abysmal failures. This disaster narrative is largely sustained by measures of quality that align neatly with wealth and position, reducing schools to competing with each other rather than being identified by their unique strengths. The most disturbing outcome of this propensity is greater inequity.

Quality-conscious middle-income parents cluster together at highly sought-after schools, seeking to better serve their own children. In the process, they not only weaken the schools they leave behind, depleting them of resources and damaging their reputations, but also, in doing so, provide a narrower educational experience for their own children. Everyone loses.

We believe it is possible to give parents richer and more useful information about schools. And, we believe it is possible to do so in a manner that builds confidence rather than eroding it. That means promoting more winners, certainly. But it also means fostering a broader sense of what it means to win, reframing our conversations about school quality to align with our true values, and laying the groundwork for an education system that is both more inclusive and more integrated.

Jack Schneider is an assistant professor of education at the College of the Holy Cross in Worcester, Mass., and the author of the forthcoming book From the Ivory Tower to the Schoolhouse: How Scholarship Becomes Common Knowledge in Education (Harvard Education Press, 2014). He can be found on Twitter @edu_historian.

Anil Nathan is an assistant professor in the department of economics and accounting at the College of the Holy Cross, where he researches and teaches the economics of education.

Vol. 33, Issue 10, Pages 21,23