Tuesday, April 10, 2012

Ri-CAN: Not Really Interested in Data

OK, so RI-CAN has a new school ranking and rating system and a website for perusing it. As far as I can tell, there are two guiding principles behind this effort:

  • Getting people to "join" RI-CAN, because you have to do that to see an individual school's scores. Expect to see a nice large number of RI-CAN "supporters" or "members" popping up in their future missives.
  • Providing some nice looking rankings for Blackstone Valley Prep, because the whole methodology seems to be built around including it as a full peer of schools which actually fully exist.

So, to accommodate BVP, for overall performance RI-CAN only uses the numbers from the current school year, in the highest grade served by the school, in reading and math (but not science and writing). This is perfectly tailored to BVP's existing data set.

I was a little surprised that they use the "testing year" numbers instead of the "teaching year," since the reform community recently realized that the teaching year data was more meaningful. However, BVP is prominently featured as having the highest African-American student performance and second highest limited English proficiency performance, but they barely make the 10% population cutoff for either in the testing year, if you round up from 9.6%, and don't have enough students still in the RI public education from the "testing year" to be eligible for either list at all.

They do helpfully note that in terms of their measurement of performance gains that:

It is important to note that this indicator is most reliable in showing a school’s impact on the change in student achievement if the school’s student population remains stable from year to year.

They do not note that this affects the reliability of some of the top (BVP) and bottom (MLK Elementary) scorers in this area.

To give credit where it is due, I do think these numbers help confirm that Deborah Gist putting her boot on the neck of RI independent charters has made their scores go way up the past couple of years. It is one of the few policies which very clearly correlates to test score changes.

RI-CAN's system is similar to RIDE's new ranking system as described in their NCLB waiver application, but rather clearly inferior (and I'm no fan of RIDE's). I'm sure RI-CAN will add more elements to their system in coming years, as BVP's data set grows. For example, right now BVP doesn't have enough special ed students to count, but if that changes, I'm sure it will be added to future rankings!

Finally, this comment by Maryellen Butke just gets under my skin:

Executive Director Maryellen Butke says another standout was the Urban Collaborative Program in Providence, a middle school that works with failing students. It had the highest test score gains for any middle school in the state - 25 percentage points.

"For them to be beating odds in that way and really making such improvement gains, I want to run right over and say tell us what you're doing, cause that's a very large gain," Butke says.

UCAP has only been around for 24 years Maryellen! It is a multi-district intervention program for middle school students who have fallen behind for one reason or another. Most are referred by guidance counselors (not lottery, it isn't a charter). Some other fun facts about UCAP:

The class size at UCAP is approximately 17 students and a special education teacher is available to assist any student needing extra help during the regular school day.

Students from grades 7, 8, and 9 are heterogeneously mixed in each class and students remain with the same teachers for their entire time at UCAP.

Students are given many opportunities for silent reading and for conferencing with a teacher or another student about things they have read or written. Students will also spend time in all classes on strategies related to problem solving. Students will be expected to understand the ideas and concepts presented in class and to demonstrate this understanding in a variety of ways. Generally, teachers do not make much use of short objective tests or other traditional methods of assessment. Instead, students will take part in projects and will demonstrate their understanding through such things as poster boards, oral presentations, research papers, creative writing, or computer projects. Also, textbooks are rarely used at UCAP. Instead, teachers create learning programs and units for students based upon the school’s written curriculum, student needs and interest, and world events.

All adolescents, particularly those who are at-risk, are faced with many challenges. With this in mind, UCAP has made the commitment to provide counseling and support for students. UCAP utilizes a case management approach under the direction of the school’s full-time, licensed social worker, and a counselor.

Perhaps RI-CAN would like to advocate for more schools like UCAP, and for all urban schools in Rhode Island to have a similar program and resources. Of course, that will never happen. It is simply not their agenda at all.

No comments: