Wednesday, July 03, 2013

2013 RIDE School Classifications: Running Your Data Through a Fuzzbox

With an electric guitar you might have a bad cable or connection somewhere, adding clicks, pops or static, or dropping out erratically. This noise is almost always unintentional and reduced as much as possible. On the other hand, you may intentionally add a fuzzbox or another effects pedal between the guitar and the amplifier. This doesn't add new noise, but it distorts, exaggerates, filters, etc. the input to create a different sound based on the source input.

RIDE's school classification system's "multiple measures" approach takes relatively clean educational data (NECAP scores, plus grad rate for high schools) and runs it through about five different fuzzboxes and effects pedals (in parallel) to create an aggregate sound/score.

For example, here's the NECAP reading and math proficiency rates added together and graphed against the single proficiency score they received in the classification system:

Obviously there is a strong correlation, since they are supposed to represent the same data, but as you can see, two schools have the same classification score for overall proficiency while being 25% different in reading and math combined (sorry Frank E. Thompson Middle School). That's not noisy input, that's fuzzed out output.

And that would seem to be the simplest component of the whole thing. It is just the proficiency rate. Everything else is some kind of more complex variation.

Your "gap closing" scores depend heavily on how many categories have enough students of each sub-type to count as a potential gap. The best scenario is to have a very homogenous small school that does well in the one or two demographic categories that count for the school. Also, it doesn't do you any good if you have uniformly high growth across demographic categories. If your students start with large gaps and they have uniformly high growth scores, you're still screwed for gap score (sorry again Frank E. Thompson Middle School).

I'm not going to dig into how the 2017 progress targets were set for each school, but... I don't even know. If a school has good overall scores and good student growth scores but lower progress scores... what is that even supposed to mean to anyone? It reminds me of something the NFL would throw in to increase "parity." Slightly hobbling schools that peaked a bit a couple years ago and boosting those that hit a little trough.

The most annoying part of the system is that you get a full rating with incomplete data. So BVP elementary is rated "Commended" with data for only 3/5ths of the categories (no growth or progress scores).

Also, schools that are rated "Focus" or "Priority" can't move out of those categories in only one year, which would make sense if the system in general had any interest in historical data, which it pointedly does not.

If I was slightly motivated, I'd try to get a spreadsheet of the data so I could at least see how closely the final ratings correlate to the simple NECAP scores. I'm sure it is high enough to wonder what the point of all this is. They are far too weird for parents or educators to use to drive selection or improvement. The deeper you dig the more you find just weird mutually-cancelling artifacts. Put another way, what would you make of Central Falls High School and E-Cubed having almost the same classification score when E-Cubed's proficiency rate in writing is almost 40% higher than CFHS's?

No comments: