Monday, April 22, 2013

Beware of Cut Scores

Jared Bernstein:

So, what’s the answer? The media gatekeepers themselves have to be highly vigilant, especially when the results they’re publishing are not transparent. (Reinhart-Rogoff)’s work, for example, involving many different countries over many years, with results put into seemingly arbitrary bins (debt/GDP<30%, 30%-60%, etc.) might be a flag that would lead a non-technical reporter to ask a lot of their sources what they thought before running with it.

An infatuation with cut scores and "arbitrary bins" should raise red flags, and it is a hallmark of the creative mathematics used by school reformers.

Or, as Stephen Downes puts it:

There's a whole industry based on evaluating teachers - and yet, it seems to me the statistics it relies upon would be dismissed outright if used to evaluate professional baseball players or other athletes.

No comments: