By Jack Schneider
Data rules in American K-12 education. In the decade since the passage of No Child Left Behind, states have assessed students in grades 3-8, as well as at least once in high school. And the result of this massive testing enterprise has been a colossal accumulation of student achievement data. Working with these raw materials, policy elites have produced a slew of indicators, including report cards on schools and, more recently, “value” added measures of teacher quality.
Yet while there is an enormous quantity of data, the qualitative picture it presents is quite limited. To their credit, analysts continue to get better at slicing the data; student growth measures, for instance, are innovative instruments that produce a level of comparability across different populations. Still, no amount of creativity can get them around the fact that they are missing most of the pieces in a large and complicated puzzle. Achievement data presents a general picture of students’ basic academic competencies. But it indicates little about a student’s ability to think or write or persuade, to perform experiments or conduct research, to paint, or to play an instrument. And it reveals nothing about a school’s climate, its academic orientation, or its culture.
Not surprisingly there has been a backlash against the testing movement. Led by vocal critics like Diane Ravitch, parents have begun to question the degree to which test scores are indicative of learning. And they have drawn increasing attention to the problems associated with a monomaniacal testing regime. That is unquestionably a good thing. But we need to ensure that the pushback against the current reform cadre’s obsession with test scores does not result in a total rejection of data. After all, good data can help us identify the strengths and weaknesses of a school, a teacher, or a student — not only for the purpose of informing parents and administrators, but also for the purpose of directing resources.
Recently a colleague and I built a school rating tool for the Boston Globe. Striving to go beyond raw test scores, we included measures like diversity, college matriculation, and school resources. We also included student growth on state tests, as well as SAT writing scores and AP scores. We openly acknowledged that we were limited by available data. And in a live online chat, we identified several measures we’d like to see systematically collected on a statewide basis — measures like teacher job satisfaction, student happiness, parental engagement, richness of art and music programming, and employee retention rates. Yet to our surprise, we still encountered some surprisingly aggressive pushback from parents and educators opposed to measurement of any sort. One parent even argued that data collection is an inherently malevolent enterprise.
It is understandable that there has been a backlash against the injudicious use of raw standardized test scores. And it is even understandable that this backlash has extended beyond those narrow measures, affecting perceptions of richer and more useful forms of data. But data collection, per se, is not the problem. Instead, the problem is rooted in the myopic approach of contemporary policy elites — an approach that has given data collection and analysis a bad name. This distinction, between what is and what might be, is an essential one that needs to be clearly articulated to educators and the public. Because, ultimately, teachers and parents — along with the young people in their charge — are the ones who have the most to gain from better data.