Thursday, July 15, 2010

One Factor in Low NECAP Scores


Of all the grades that are regularly tested as part of the congressionally mandated NAEP program, the 12th grade results have long been the most disappointing. That has led some experts to wonder whether the problem stemmed from poor quality instruction in high schools or whether the older and more-savvy high school seniors just weren't trying as hard as the younger test-takers. ...

In the end, the study found, both of the monetary incentives spurred students to do better than they might have otherwise, although the second condition, in which part of the payout hinged on the students getting answers correct, proved to be the stronger incentive. Under both conditions, though, scores for both male and female students were, on average, at least 5 points higher than the scores for the no-incentive group.

This study is asking a different question than Roland Fryer's recent work on incentives -- this one is just about whether or not incentives make kids try harder on the day of the test. Anyone who has proctored low-stakes exams for inner city high school students knows just how big this issue is; in many cases the kids just put their heads down after five minutes. And why shouldn't they? It doesn't affect their lives.

This doesn't impact state to state comparisons on NAEP, but it does have an impact if you tried to correlate state assessments between states that have differing "stakes" attached to test scores. Up to this point, all standardized achievement tests, including NECAP, in RI have been no-stakes for the kids.


Jason said...

This is definitely one area we agree. I commented about this over at the Core Knowledge Blog.

A small think worth noting, that I think you're getting at here, is that motivation varies across age levels and "geography". I would guess that the low-stakes of NECAP adds a negative bias to the scores of older kids and kids from districts that are historically low performing.

The situation is bad in schools with sub-10% proficiency in 11th grade math, but the numbers may be exaggerating the story.

Tom Hoffman said...

See also the comments here:

Jason said...

Thanks for the heads up. I actually took a look at that exam in April. I posted some comments on the older post.