Monday, April 01, 2013

The Validity Question

Tom Sgouros:

What does (Peter Merenda, who literally wrote the book on Educational Measurement) think of my critique of using the NECAP test as a graduation requirement? My suggestion was that the test is created with the expectation that lots of students will flunk, for perfectly valid statistical goals. I am deeply chagrined to say that he chided me… for not going nearly far enough. He said my critique was correct as far as it goes, but there is a far worse problem: validation. Those markups on the paper above say things like, “RIDE as user of NECAP is in violation of National Testing Standards“, and “the test scores have not even been validated for any purpose.”

Validating a test means to ask in a serious and disciplined way, what does the test actually measure? It usually means stepping outside the framework of the test itself to see how good the correlation is between test results and whatever it is you want to be measuring. For an employment test, you might try to compare job performance with test results (before making your hiring dependent on those test results, that is). For an intelligence test, you might compare test results with some other intelligence test. And for a graduation test, you might want to examine the test-takers and see, in some independent way, whether the students who pass deserve to graduate and whether the students who flunk do not.

In a related vein, Politifact:

CCRI uses a placement test called Accuplacer, developed by the same group responsible for the Scholastic Assessment Test for college-bound high school students, to identify students who need remedial work.

Among those 2,250 recent high school graduates, just over 75 percent needed some type of remedial course, whether it was math, English or writing, LeBlanc said. "That's the highest we've had in awhile." The rate usually ranges from 70-75 percent. He said that's not unusual for a community college.

The biggest need was usually for remedial math -- 65.7 didn't pass the math requirement. They had to take Fundamentals of Math -- which deals with basic arithmetic, percentages and working with fractions -- or high school algebra.

If that's the issue, maybe we should be giving high school students the Accuplacer. I bet it is easier than the NECAP.

And... the College Board will be happy to help you prepare for ACCUPLACER and apparently students will be able to use ACCUPLACER as a NECAP replacement in the graduation requirements.

No comments: