Tuesday, December 11, 2012

PIRLS 2011: We've Succeeded at What We've Truly Focused On

From where I sit, we've only truly had an all hands on deck, we're going to spend whatever needs to be spent, this nation is at risk, I'd consider it an act of war if imposed upon me by an unfriendly foreign power, kind of response in one sliver of education reform over the past 15 years or so, and that is in elementary reading. I'm not going to marshal statistics to prove it; it should be obvious to anyone with a good view of the process.

To me this is symbolized in the elementary school down the street where the new literacy/data center sits in the ruins of the formerly state of the art science center (itself within the original library).

That is not to say I agree with a lot of our literacy strategy, but there is a vast difference between the amount of push in early reading instruction now and decades earlier.

And, importantly, we've traditionally had good professional capacity in this area.

So... the scores for the 2011 PIRLS 4th grade international reading test is out, and we killed it. Killed it.

These two graphs lay it out pretty well. First, the overall score distribution (red = 5th-25th and 75th-95th percentile, black = 95% confidence interval for the average). Click to expand:

When you break out the scores by economic background, we do even better:

There's a name for the effect when you outscore everyone else in each subcategory but get a lower overall score because of the relative distribution of each group (i.e., you have too many in the most disadvantaged group). I'm not going to look up the name right now, but that's what's going on here. We're #1 in the top two groups and 5th in the third, by my scan. Against Finland we're +15/+3/-4. Against Russia we're +15/+8/-12. Against Singapore we're +1/+14/-4. We just have a low number of "more affluent" schools and a lot of "more disadvantaged" ones.

I guess Florida is the only state that took the PIRLS to be scored individually, and their scores are gaudy. First, first, and second by income group.

Yes, I know Florida's 4th grade reading scores are suspect because of their retention policies, but the whole northeast does as well or better than Florida on 4th grade NAEP, so some states would do even better, and Rhode Island might even look pretty good.

Ultimately, no, I don't put that much stock in any one test, etc., and we know our scores tend to decline at higher grade levels, and we don't do as well in math and science.

But, this is still an impressive indication of what we can do if we set our minds to it and invest real money and effort -- without the whole privatization agenda. I don't see us beating the world in math for pretty much the same reason we won't win the World Cup anytime soon; we do pretty good in soccer, for a country that doesn't really like soccer that much. And I'm not convinced we're even really trying in science at this point (heck, it's only 1/4th of STEM).

But Jeez Louise, this is probably the best we've ever done on any international test in K-12, ever. You'd think people would be happy. Especially the people who profess to think international comparisons are extremely important.


luke hodge said...

It does appear that the US does well, but I am not sure that the chart by economic level is an apples to apples comparison. The international average is 35% affluent, 35% middle, 30% disadvantaged. The US claims to be 18% affluent, 31% middle, 51% disadvantaged. I am sure we have a higher child poverty rate than most of the high performing countries, but that distribution seems a little suspect.

luke hodge said...

What is interesting is how the principals report on the economic group for their schools. In the US the reporting is 18% affluent, 31% middle, & 51% disadvantaaged. The international average is 35%, 35%, 30%. I don't doubt that we have more child poverty than the high performing countries, but this does not appear to be an apples to apples comparison - 68% of Russian schools were categorized as affluent for example.

Tom Hoffman said...
This comment has been removed by the author.
Tom Hoffman said...

Yes, I wouldn't try to take too much subtle meaning from this, but we are first in the high and middle categories, which is hardly consistent with a systematic crisis.