Thursday, July 22, 2010

Catching Up to Tuttle SVC on HCZ Analysis

I'd just like to note that I was over a year ahead of the Russ Whitehurst in generating a contrarian reading of Dobbie and Fryer's analysis of the Harlem Children's Zone and its schools.

Me, May 8, 2009:

In particular, Fryer's self-described epiphany, "The attached study has changed my life as a scientist," is not credible. The statistics from the study, which is not linked, show significant progress by middle school students at The Promise Academy Middle School, but on the surface at least, little different than what other high achieving middle schools in New York score on common assessments. Frankly, KIPP, Democracy Prep and some of the other charters bump up against the statistical upper limits of the assessments used by the city and state. It would be difficult mathematically to score significantly higher.


OK, I just read through Are High-Quality Schools Enough to Close the Achievement Gap? Evidence from a Bold Social Experiment in Harlem. By my reading, the key point is that based soley on high-stakes English and math tests, the only interventions in the Harlem Childrens' Zone that have a substantial effect are the schools. What makes this study different than similar ones is that they can compare the scores of kids from inside the zone and outside, and find that there is not a significant difference.

I'm not enough of a statistician to say whether or not the amount of growth shown at the Promise Academy schools is significantly greater than that at older schools which Promise Academy explicitly emulates. If it is true, we're left with a mystery. What are they doing differently? Fryer claims to show it is not because of the other measures in the Zone.

If you buy this study, it is an argument against replicating the Children's Zone.

A couple other points:

  • Whitehurst says three higher scoring KIPP schools "None provide or depend on community and social services to achieve their academic mission." Has this been documented by observation? Or is it just conventional wisdom? Where do you draw the line between "no excuses" and "social services? If a KIPP counselor does a house visit is that a social service? If a teacher dispenses advice at 9:00 PM on their cell phone to a student is that a social service? If a teacher refers a parent to a free clinic is that a social service? What if a city or HCZ social worker does exactly the same things?
  • Also, these social services are provided to the non-HCZ students in HCZ schools:

    The schools provide free medical, dental and mental-health services (students are screened upon entry and receive regular check-ups), student incentives for achievement (money, trips to France, e.g.), high-quality, nutritious, cafeteria meals, support for parents in the form of food baskets, meals, bus fare, and so forth, and less tangible benefits such as the support of a committed staff. The schools also make a concerted effort to change the culture of achievement, surrounding students with the importance of hard work in achieving success. These types of school policies are consistent with those that argue high-quality schools are enough to close the achievement gap.

    I suspect that this level of social services may be greater than those received by non-HCZ charter school attending residents of the Zone, confounding the entire analysis.

  • Despite the hype associated with the Harlem Children's Zone, which is at least an impressive feat of philanthropic marketing, the total scope of benefits may be simply inadequate. Compare the quality of life of a HCZ resident to that of any urban neighborhood in any Nordic country.

1 comment:

Jason said...

Solid post with some good stuff. Sara Mead made a few of these points yesterday and also has a pretty good take on this stuff that you may be interested in: