Thursday, March 15, 2012

Could You Make that Less Sincere?

Jonathan Alter:

“We’ve spent $300 million in this country on teacher-effectiveness research, and what turns out to be the best predictor?” asks Timothy Knowles, who runs the Urban Education Institute and headed up Rahm’s transition team on education. Knowles offers me a pleasantly contemptuous “Hmmm” and answers, “It’s students.” Their evaluations of teacher quality are surprisingly accurate when correlated with other measurements. Standardized tests, he says, “have been gamed so mercilessly by many states that they’re of limited use.” Responding to the report cards was voluntary this year. Rahm has ordered that compliance be mandatory for Chicago schools in 2012–13, which means that every school in the city will for the first time be thoroughly evaluated.

Without even noticing that Jonathan Alter wrote this peice for The Atlantic, that paragraph jumped out at me as a perfect example of our skewed, insincere school reform discourse. To fisk:

  • "We’ve spent $300 million in this country on teacher-effectiveness research." Perhaps this refers to Gates' MET project. Or the feds' TIF? Either way it is probably either too high or too low. Too high if you're counting money spent on research itself, too low if you're talking about various ongoing schemes to improve teacher effectiveness that are both very expensive and are whose results are being researched.

    But rhetorically, why throw in that big handwavy number at all? Just a couple years ago when MET was being launched, we were being told teacher effectiveness was a vast undiscovered country. This was bullshit then, and it is bullshit now when Knowles tries to impress you with a nine-figure budget, trying to signal to the reader that now the problem is well understood, and here's the simple, common-sensical answer.

    The reader, and Alter, are likely to think that what follows is the consensus of a large body of rigorous research; in reality it is probably just an inaccurate gloss on the preliminary results of a single initiative.

  • "what turns out to be the best predictor?” asks Timothy Knowles, who runs the Urban Education Institute and headed up Rahm’s transition team on education. Knowles offers me a pleasantly contemptuous “Hmmm” and answers, “It’s students.” Their evaluations of teacher quality are surprisingly accurate when correlated with other measurements. Standardized tests, he says, “have been gamed so mercilessly by many states that they’re of limited use.”" First off, this overstates the evidence I can find on the efficacy of student surveys. On a deeper level, this claim is nonsense because there is no platonic ideal of effectiveness that we can use to confirm the accuracy of any given measure. We can see if the measures correlate with themselves over time, and with other measures, and in the studies Knowles is almost certainly referring to, the ultimate measures are test scores, which Knowles quickly pivots to discredit because they've been "gamed."

    But the MET preliminary report actually defends the validity of standardized tests anyhow. And if tests have been gamed, why wouldn't student surveys be?

    Finally, it isn't like these guys are interested in the opinion of students (or parents or teachers) in general. Just when it serves their larger purposes, which apparently it does now.

Now, Knowles doesn't work for J.C. Brizard in the CPS, but still, just three paragraphs later we get this:

Shortly after arriving, Brizard informed his principals, who every year had rated 99 percent of Chicago teachers “superior or outstanding,” that they must change performance standards faster. “We’re getting better. We moved from less than 1 percent to 1 percent ‘unsatisfactory,’” he told them wryly. He recommends that teachers read or watch the video of Doug Lemov’s Teach Like a Champion, the bible for reformers who stress great teaching. Brizard understands that many charter schools fail, and that traditional schools cannot all adopt the crushing teacher workloads of the charters that succeed.

Wait, what happened to the awesome accuracy of student surveys? What happened to $300 million in research? Why are we reading a book that's just a collection of practices from a selection of successful schools? We've had those for years without spending $300 million (and without fixing American education). And how does a "crushing teacher workload" fit into this? Can't we just hire more teachers?

It just doesn't seem to matter what people are actually saying, it all just comes out as "yadda yadda reform" anyhow.

No comments: