The Reinhart/Rogoff Episode Shows Us Why Data Openness is So Important

Get your news from a source that’s not owned and controlled by oligarchs. Sign up for the free Mother Jones Daily.


What’s the most important takeaway from this week’s kerfuffle over the errors in the Reinhart/Rogoff dataset? Obviously there are public policy implications, since R&R concluded that debt levels above 90 percent of GDP lead to a sharp dropoff in economic growth. This provided important intellectual support during the past few years for the belief that it’s more important to focus on deficit reduction than on unemployment and fiscal stimulus.

In the end, though, although R&R’s results were important, they probably weren’t crucial. As Jared Bernstein notes, deficit hawks were “using research findings the way a drunk uses a lamppost: for support, not for illumination. If the R&R lamppost turns out to be wobbly, the austerions [] will find another one. In this town, I’m sorry to say, you can pretty much go think-tank shopping to buy the result you seek.”

I agree. R&R’s results were convenient to have, but not crucial. People who are dedicated to cutting spending, either for political or ideological reasons, would simply have trotted out other justifications if R&R’s study hadn’t been available.

More important, then, may be the light this shines on the fact that an awful lot of research is based on datasets that are kept private. Megan McArdle pinpoints both the reason for this and the dangers involved:

There are a lot of private data sets out there these days, and a lot of work being produced off of them. Why can’t more of us see it?

Mostly, I suspect, because of the economics of the thing. Assembling a nice private data set is a huge amount of work. You want to be able to mine that work for publishable insights. Very little professional credit accrues to the guy who built a great dataset which everyone else uses to generate elegant new findings. The credit goes to the authors of the elegant new findings. Which means that once you’ve built a dataset, you want to keep the thing to yourself as long as possible.

….Unfortunately, there’s always the possibility that “I want to hold onto it as long as possible so I can publish” can be a cover for “I need to hold onto it as long as possible because if anyone else sees it, it will rapidly become obvious that my results aren’t very robust.” Or, in some cases, for “There are no results because I made the whole thing up.”

This was an issue that was front and center during Climategate. Climate skeptics were unhappy that the raw data collected by various research groups (mostly using public money) wasn’t made available to them, and they made the reasonable point that if your analysis is correct, you shouldn’t be afraid to share the underlying data. Professional researchers, for their part, were reluctant to make their data available because they knew it would generate an enormous flurry of amateur debunkings that they’d have to waste their time picking apart. In the end, though, releasing the data didn’t make much difference because it turned out the professionals were interpreting it correctly.

This is why the R&R incident might be more important for the cause of openness. It’s not just that critics found errors in the dataset once they got hold of it. Even more important might be the fact that once the raw data was available, they were motivated to do certain types of analysis that R&R didn’t do. For example, they discovered that R&R’s results were quite fragile, changing dramatically with the addition or subtraction of just a few observations. They also found that, far from showing a sharp drop in growth when debt passes 90 percent of GDP, the data actually showed a sharp drop between 0-30 percent of GDP and only a rather leisurely decline above 70 percent. Finally, and perhaps most important, UMass economist Arindrajit Dube took a closer look at what the data said about causality, and produced persuasive evidence that R&R’s own data suggested not that high debt leads to low growth, but that low growth leads to high debt. R&R could have performed the same analysis, but for some reason they weren’t interested in it.

As long as the dataset was private, nobody would ever have known any of these things. We could have guessed them—and a lot of people did—but we wouldn’t have had any evidence. Now we do. Unfortunately, as McArdle says, “the incentives are all wrong for data openness.” This episode suggests why it might be worthwhile to try to change those incentives.

BEFORE YOU CLICK AWAY!

“Lying.” “Disgusting.” “Scum.” “Slime.” “Corrupt.” “Enemy of the people.” Donald Trump has always made clear what he thinks of journalists. And it’s plain now that his administration intends to do everything it can to stop journalists from reporting things they don’t like—which is most things that are true.

No one gets to tell Mother Jones what to publish or not publish, because no one owns our fiercely independent newsroom. But that also means we need to directly raise the resources it takes to keep our journalism alive. There’s only one way for that to happen, and it’s readers like you stepping up. Please help with a donation today if you can—even a few bucks will make a real difference. A monthly gift would be incredible.

payment methods

BEFORE YOU CLICK AWAY!

“Lying.” “Disgusting.” “Scum.” “Slime.” “Corrupt.” “Enemy of the people.” Donald Trump has always made clear what he thinks of journalists. And it’s plain now that his administration intends to do everything it can to stop journalists from reporting things they don’t like—which is most things that are true.

No one gets to tell Mother Jones what to publish or not publish, because no one owns our fiercely independent newsroom. But that also means we need to directly raise the resources it takes to keep our journalism alive. There’s only one way for that to happen, and it’s readers like you stepping up. Please help with a donation today if you can—even a few bucks will make a real difference. A monthly gift would be incredible.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate