Yesterday I wrote a gloomy post about education, suggesting that despite decades of punditry that identify heroes and villains with abandon, “You can go down the list of every ed reform ever touted, and they either can’t scale up, turn out to have ambiguous results when proper studies are done, or simply wash out over time.” I’ve been meaning for a while to post something else on this subject, because Bob Somerby has been beating this drum for a while. Here’s Bob:
NAEP scores for black kids and Hispanic kids are way up, in reading and math, in the past dozen years….But so what? Everyone agrees to avoid discussing these large score gains, even as we wring our hands about school reform and savage America’s teachers, along with their infernal unions. We think we understand Kevin’s view of this matter; a few months ago, he said those score gains don’t mean all that much because they haven’t been matched in NAEP testing at the 17-year-old level. (In that sense, they might be said to “wash out over time,” although the kids recording
the higher scores haven’t turned 17 yet.) That said, the score gains at earlier ages are very large — and yet, we all agree to ignore them, seeming to find them unworthy of exploration.
NAEP scores are widely considered pretty reliable, and we’ve been conducting NAEP tests for several decades. So what do they show about gains among black kids? The basic chart is on the right. The data comes from this report, which provides both reading and math scores for black, white, and Hispanic kids at three different age levels since 1971.
I don’t really want to spin this any particular way. I just want everyone to see the basic data so we’re working from similar baselines. Roughly speaking, there were substantial gains at all grade levels between 1971 and 1988. After that, there was a decade of stagnation.
Then, after that, there was a decade of qualified progress. I say “qualified” because it’s hard to draw any firm conclusions from it. Since the late-nineties, 9-year-old black kids have improved their reading scores 18 points. Ten points on the NAEP is roughly a grade level, so that’s pretty impressive. 13-year olds improved nine points. 17-year olds improved two points.
What does that mean? I just don’t know. The gains among 9-year-olds are genuinely extraordinary (white kids also improved during this period, but only by seven points). Two grade levels is a huge difference. But half the improvement washed out in just the next four years. And in the four years after that the rest of it washed out. Reading scores among 17-year-old black kids have been flat for two decades. Ditto for white and Hispanic kids.
Now, this doesn’t mean that whatever happened in 1999 doesn’t matter. School reforms generally start at the elementary level and work their way up. So maybe we just haven’t had time for the reforms of the past decade to show up among 17-year-olds. It’s also true that dropout rates confound the data gathering at higher age levels.
But I’d be lying if I didn’t say that I’m skeptical. It’s true that there’s some evidence in the data that the gains of the 70s and 80s were staggered: age 9 first, then age 13, then age 17. But only partly. Generally speaking, all three cohorts progressed at the same time. This time around we aren’t seeing it. 9-year-old black kids improved their reading scores by a stunning 14 points in the single period between 1999 and 2004. These are the 13-year-olds of the next testing period, but 13-year-olds only gained three points between 2004 and 2008.
So…..I’m not sure where to go from here. Bob is right that test scores1 among 9-year-old black kids have made impressive progress over the past decade, and this gets largely ignored by a media that seems interested only in dramatic tales of heroic reformers and evil teacher’s unions. On the other hand, none of this progress matters much unless those improvements are persistent. If the gains all wash out by the time kids graduate from high school, they haven’t done any good.
So that’s the data. It shows what it shows, and the story is partly encouraging and partly not. It’s also unfinished (data always is), and in any case, I don’t know of anyone who’s adequately explained why we’ve seen such impressive elementary school gains over the past decade. Without that, we hardly know what our next steps ought to be. Caveat emptor.
1Math scores have shown similar patterns: big gains at age 9, smaller gains at age 13, and very small gains at age 17.
UPDATE: Chart corrected to show the time series data more accurately. Thanks to Chad Orzel for the Excel lesson.
