Here’s a Textbook Criticism of Econometrics Gone Bad

A couple of weeks ago I linked to a study that suggested the anti-overdose drug Nalexone might, on net, actually cause more opioid deaths. I was skeptical because the data seemed to show no real pattern. The authors drew a dashed red line at the point where Nalexone was legalized, and sure enough, it looked like opioid deaths jumped after the new laws were passed. But the line seemed entirely arbitrary.

Now Andrew Gelman has weighed in. He has pretty much the same criticisms I did, but he also has the statistical chops to be taken seriously. However, he also links to Richard Border, who doesn’t say a word but lets the numbers speak for themselves:

This is a textbook criticism of statistics based on too few data points. As Border shows, you can put the cutoff line anywhere and it looks like there’s a big jump in opioid deaths. But that doesn’t mean anything. The error bars are enormous, and the before-and-after “trends” are meaningless.

I’d add that this also shows the value of simply using common sense. If you remove all the lines and the error bars and simply look at the scatterplot, there’s nothing there. That should make you skeptical from the very start. It’s true that occasionally you can tease real results out of data like this, but you’d better be damn sure of what you’re doing and you’d better check it ten different ways. The authors of this study didn’t do that.

One More Thing

And it's a big one. Mother Jones is launching a new Corruption Project to do deep, time-intensive reporting on the corruption that is both the cause and result of the crisis in our democracy.

The more we thought about how Mother Jones can have the most impact right now, the more we realized that so many stories come down to corruption: People with wealth and power putting their interests first—and often getting away with it.

Our goal is to understand how we got here and how we might get out. We're aiming to create a reporting position dedicated to uncovering corruption, build a team, and let them investigate for a year—publishing our stories in a concerted window: a special issue of our magazine, video and podcast series, and a dedicated online portal so they don't get lost in the daily deluge of headlines and breaking news.

We want to go all in, and we've got seed funding to get started—but we're looking to raise $500,000 in donations this spring so we can go even bigger. You can read about why we think this project is what the moment demands and what we hope to accomplish—and if you like how it sounds, please help us go big with a tax-deductible donation today.

We Recommend


Sign up for our newsletters

Subscribe and we'll send Mother Jones straight to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.


Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.