Friday, November 19, 2010

This Doesn't End Well


Under the proposal, starting with the Class of 2012, students would be eligible to receive one of three diplomas, based on their scores on standardized tests in English and math, provided they also complete the required coursework and a portfolio or senior project.

Students take the tests, called the New England Assessment Program, in October of their junior year. The tests are challenging: Scoring proficient on the math portion, which covers algebra, geometry, statistics and probability, is roughly equivalent to scoring a B+ or better, according to state education officials.

Students who score “partially proficient” on the tests would be eligible for a “Rhode Island” diploma. Students who score “proficient” would receive a “Regents” diploma. Students who score “proficient with distinction” on the tests and do honors-level work across the board would receive an “Honors” diploma.

Students who fail to score at least partially proficient on the tests would be required to re-take the tests in their senior year. And schools would be required to offer additional supports along the way.

If a student fails to improve between junior and senior years, they would not be eligible to receive a diploma and would instead receive a “certificate.”

Setting aside a more substantive discussion of both tiered diplomas (which I regard as necessary in general) and the role of standardized tests in the process (which I think should be minimized), let's just look at the numbers here and think about the politics. Below is the percent of students who would be elegible for each level of diploma per test, based on the 2009 testing year data (RI=state, PVD=Providence, FHS=your favorite high performing high school closed for low performance):

Reading and writing are in the ballpark of the right spread of scores, with the idea probably being between the two. Math, not so much. Now, RI underperforms other states in math, but Vermont doesn't, and by this standard, 37% of all Vermont juniors and 56% of Vermont juniors eligible for free and reduced lunch would not qualify for a RI diploma. Whether or not this test in general is appropriate to the task, the current cut scores almost certainly are not.

And of course, if you have to pass all of them at the correct level to get credit, that cuts things down further (I can't find a detailed description of the proposal online).

I'm hard pressed to figure out who the in-state constituency for this decision is (the out of state audience is more obvious). To conservatives the multi-tier system looks like a cop-out; the direct and immediate problems are of course obvious to low income students, parents and their teachers; but beyond that, creating an "honors" diploma dependent almost entirely on a math test given at the beginning of 11th grade is likely to make many a competitive head explode in the suburbs. Your 2012 valedictorian may already have blown it.

Beyond that, while RI's large diversity of school designs serving low income students has produced a wide range of performance in the reading and writing NECAP's and other measures, nothing seems to make much of a difference in math scores, and it is difficult to imagine neighborhood schools under the best circumstances moving things more than, say, 20 points in the right direction.


Jason said...

I'm not sure if it's an indictment of the NECAP Reading/Writing or NECAP Math test but it's very bizarre that we've seen so much success in schools producing great reading scores instead of math.

Most of the research out there shows that the "math" gap is easier to close with low-ses and ELL students (or at least, more frequently we see positive results of interventions in math than in English). Very few program evaluations have found remarkable effects in ELA, and almost always, the effect on math is higher even in those cases.

Tom Hoffman said...

I'm not sure it is an indictment of either as much as a function of where one places the cut scores and how aspirational the designer's goals are.

I agree that elementary and probably particularly middle (given the spreads by that point) math scores are easiest to boost, but I haven't seen many examples of that at the high school level -- particularly in schools that don't also have high attrition.

Anonymous said...

Of what possible benefit is it to grant multiple types of diplomas?

Tom Hoffman said...

Given that we historically have multiple objectives for high school students -- e.g., qualified for boot camp, factory or waitressing vs. "college and career ready," just to look at one facet -- I think you can make an argument for two levels reflecting that dichotomy. But it isn't a slam dunk, and this particular three or four tier proposal is just nuts.

Anonymous said...

But don't grades/transcript take care of that? I mean, is a college (other than one in RI) really going to spend the time to figure out what the various diplomas mean? Is an employer? Would either care? I doubt it.

On the bright side, it sounds like at least a couple of positions at the state board of ed to cook this up and administer ;)

Tom Hoffman said...

Yeah, the current Regents may be accelerating their departure with this.

Jason said...

The conversations around this issue have been interesting to watch/listen to over the last 3 or 4 months. I think the bottom line is the regents are determined to place system-pressures on districts that aren't doing their job when it comes to developing a PBGR approved system. Many of these "tiers" collapse once a system is approved.

The issue seems to be that many districts are not yet prepared to be compliant which involves a series of systems and interventions being in place to ensure that all kids have supports to make the path to graduation viable for all students. They also need to do some work developing assessments outside of NECAP which are high-quality, reliable indicators of performance-based achievement or portfolios with a solid assessment system. Though the districts have known about this shift for at least four years, about half are still not approved.

The whole PBGR regulation is about placing pressures on schools to make improvements to the structural problems which prevented persistence and led to increased drop outs. Some quick changes to the regs and in voting at the very last stages of the PBGR development placed pressures on kids as well as systems. Having both simultaneously is messy and hard.

In the end, is the diploma a signal that you have learned a certain set of material, had a certain amount of seat time, or either of those two in combination with attendance within a system that competently guided and supported your completion? Not answering this question has led to an overly complex stop-gap that is hoped to simplify quickly as more districts come into compliance.