Big Surprise: Yet Another Ed Reform Turns Out to be Bogus

Do high schools with higher standards get better performance from their students? If you require everyone to take college prep classes, will more kids go to college? The San Jose school district has long been a poster child for this notion, but guess what? It turns out it was all a crock:

San Jose Unified has quietly acknowledged that the district overstated its accomplishments. And a Times analysis of the district’s record shows that its progress has not, in fact, far outpaced many other school systems’….In 2000, before the college-prep program took effect, 40% of San Jose graduates fulfilled requirements for applying to University of California and Cal State University. In 2011, the number was 40.3%.

My cynicism about the ed reform community grows by leaps and bounds every time I read a story like this. And that’s pretty often. Here’s my advice for what you should do whenever you read an article about a school that’s shown miraculous results by applying some reform or another (or by hiring a miracle worker of some stripe or another):

1. Don’t believe it if it’s based on a single school or other small sample.
2. Don’t believe it if most of the evidence comes from the school itself.
3. Don’t believe it if the reform in question was put in place only a few years ago.
4. Don’t believe it if it hasn’t been replicated elsewhere.
5. Don’t believe it unless it’s been rigorously tested by academics who didn’t already support the idea in the first place.
6. And even if it passes all those tests, don’t believe it anyway.

The number of ed reforms that hold up when the evidence is looked at critically seems to be tiny. The number that continue to work when they’re scaled up seems to be tiny. The number that continue to show results all the way through high school seems to be tiny. The number that can withstand critical scrutiny seems to be tiny. And of the ones that are left, the cost to keep them up usually appears to be prohibitive.

I understand that I’m being too cynical here. I’m probably going to get the usual batch of emails from ed reformers telling me that there are too reforms that really and truly work. And I suppose there are. But I don’t think you can go too far wrong by being almost boundlessly and annoyingly skeptical about this stuff. Don’t worry about seeming unsophisticated. Just keep repeating that you don’t believe it until and unless the evidence becomes simply overwhelming. You won’t go too far wrong with that attitude.