MT: What is your advice on the right balance between validity and reliability, especially if we want to embrace the goals implicit in the Common Core?
HE: I think the importance of reliability has been overblown.
BL: I agree it is less important than comparability and validity and fairness. It would be highly desirable to go where the two state testing consortia want to go. They want to include, in addition to multiple-choice items, items where kids are required to do things, solve problems and show how they come up with solutions to the problems they are given. But the realities of timing and cost are pushing them in a direction that will likely force them to come up short.
MT: Is this country getting ready to make a profound mistake? We use grade-by-grade testing in grades 3-8 but no other country is doing it this way for accountability; instead they test 2 or 3 times in a students' career. If the United States did it that way, we could afford some of the best tests in the world without spending any more money.
BL: Raising the stakes for our test-based accountability systems so that there will be consequences for individual teachers will make matters even worse. Cheating scandals will blossom. I think this annual testing is unnecessary and is a big part of the problem. What we should be doing is testing at two key points along the way in grades K-8, and then in high school using end-of-course tests.
HE: I am in the same place as Bob. The multiple-choice paradigm first used in WWI and eventually used to satisfy the NCLB requirements has proven to be quite brittle, especially when applied in every grade 3-8 and used to make growth assumptions. The quick and widespread adoption of multiple-choice testing was in hindsight a big mistake for this country, but—now — states will tell you it is all they can afford.