Wednesday, July 28, 2010

This is the Way the NECAP Works, Too


The scaling approach taken by the DC CAS is, to my mind, pretty unconventional, because the scaled scores do not overlap across grades. In grade four, the minimum possible scaled score is 400, and the maximum possible scaled score is 499. In grade five, however, the minimum possible scaled score is 500, and the maximum possible scaled score is 599. (The same approach is used in grades six through eight.)

This means that a fourth-grade student who got every question on the fourth-grade math assessment correct would receive a lower scaled score than a fifth-grade student who got every question wrong on the fifth-grade assessment.

That sounds ridiculous, but it’s not problematic if the scale for fourth-grade performance is acknowledged to be different from the scale for fifth-grade performance. The design of the DC CAS allows for comparing performance in fourth grade in one year with fourth-grade performance in the next year; but it doesn’t permit measuring how much students have gained from one grade to the next. Measuring growth from one grade to the next requires a test that is vertically equated.


Jason said...

Not sure why they would calculate things as Pallas writes. However, that is not to say that NECAP is not vertically equatable (I'm not sure that it is).

Simply making the scaled score ranges different for each grade does not mean that they cannot be related to one another. If the scaling is at least in part normalization (and I would hope that it is), then the difference between a student who scored 400 and 401 is the same as the difference between a student who scored 500 and 501. 450 to 451 would be the same as 550 and 551.

In this case, we're just looking at the same exact scale from 0-99 with a number in front to indicate grade-level.

I don't know if this is how it works, but I would hope it does.

Tom Hoffman said...

Hope is not a plan.

Jason said...

Btw it's not vertically aligned at all and the scores are not similar. I confirmed that. That means that calculating growth in RI is not impossible, but quite a bit more complicated.

Tom Hoffman said...

It is also one reason why you won't see me spouting off with ad hoc analyses of elementary school scores!

Jason said...

Tom Hoffman said...

I look forward to Mathematica and DCPS publishing their secret formulas.