Friday, June 11, 2010

Giving 'em a Taste of Their Own Bullshit

Christopher Bergfalk:

In NYC Chancellor Joel Klein’s 4/9/10 article in the Washington Post, ”Why great teachers matter to low-income students” he states when referring to the NAEP test that “Ten points approximates one year’s worth of learning on these national tests.”* He goes on to take the 33 point difference in avg. scale score for “poor black students” in Boston vs. those same students in Detroit to mean “by fourth grade, poor African American children in Detroit are already three grades behind their peers in Boston.”

What does this means in DCPS as evidenced by the discrepancy among average scale score increases on the 2009 8th grade NAEP Math test for the DCPS demographic groups? According to Chancellor Klein’s methodology, it means:

  • In 2007, the difference in average scale score between non-economically disadvantaged students and economically disadvantaged students was 21 points or 2 years worth of learning. In 2009, it is 33 points or 3 years of learning. Those left behind are now an additional year behind in 2009.
  • In 2007, the difference in average scale score between economically disadvantaged Hispanic and Black students was 7 points, less than a year of learning. In 2009, it was 22 points or 2 years of learning. Those left behind are now further behind their peers.

* 10 points on NAEP tests is equivalent to 1 year of learning can also be found on the “McKinsey & Company” April 2009 “Detailed Findings on the Economic Impact of the Achievement gap in America’s schools.”

I'm not a psychometrician, but I have a strong suspicion that the 10 points = 1 year bit is a complete crock. And based on the high degree of rigor in the rest of Chris's analysis of DC's NAEP scores, I'd suspect he thinks so too. Tactically, though, if you're going to win, you gotta be willing to use the other side's tricks against them, explaining exactly what you're doing in the process. And hopefully having fun, too.

His larger point in the presentation underscores what complex dynamic systems school districts are, especially when you're shifting around highly segregated populations across district, charter and private schools that may be opening, closing and re-organized at an ever-accelerating rate. Shifts in population may completely mask gains and losses in teaching and learning, by accident or design.

No comments: