Here’s Why All the Polling Models Are Probably Right

For indispensable reporting on the coronavirus crisis and more, subscribe to Mother Jones' newsletters.

Paul Waldman describes the Bizarro-world war on polling guru Nate Silver that’s gained steam over the past week:

In the last few days, we’ve seen a couple of different Silver narratives emerge as attention to him has increased. First, you have stories about how liberals are obsessing over Silver, “clinging” to him like a raft in a roiling sea of ambiguous poll data. Then you have the backlash, with conservatives criticizing him not because they have a specific critique of the techniques he uses, but basically because they disagree with his conclusions….Then you’ve got the reporter backlash. At Politico, Dylan Byers raised the possibility that Silver would be completely discredited if Mitt Romney won, because “it’s difficult to see how people can continue to put faith in the predictions of someone who has never given that candidate anything higher than a 41 percent chance of winning.”

This whole thing is deeply weird. Could Nate be wrong? Sure. Ditto for Sam Wang and Drew Linzer and all the rest of the poll modelers out there. But if they’re wrong, it probably won’t be because of their models. After all, with minor differences they all do the same thing: average the publicly available state polls, figure out who’s ahead in each state, and then add up the electoral votes they get for each candidate. Sure, they all toss in a little bit of mathematical secret sauce, but not really all that much. You could do the same thing if you felt like it. Want to know who’s ahead in Ohio? Go add up the five latest polls and then divide by five. Voila. You are now your own Nate Silver.

Needless to say, though, the poll modelers are only as good as the polls they use. If the pollsters are systematically wrong, then the models will be wrong. And while there are a few small sources of potentially systematic bias (not calling cell phones, demographic weighting, etc.), by far the biggest is the pollsters’ likely voter screens. But even here, with one or two exceptions, this is pretty simple stuff. Most pollsters just ask a question or two that go something like this:

  • Are you planning to vote?
  • How sure are you that you’ll vote?
  • Really? Honest and truly?

That’s about it. If you tell them you’re highly likely to vote, they mark you down as a likely voter. If not, they don’t. There’s no rocket science here.

So if the modelers are wrong, it will probably be because the pollsters were systematically wrong. And if the pollsters are systematically wrong, it will probably be because this year, for some reason, people started lying about their likelihood of voting. And while anything’s possible, I sure can’t think of any reason why this year there would be a sudden change in how truthful people are about their intention to vote.

That’s what this whole controversy comes down to. Conservatives seem to be convinced that Democrats simply won’t turn out in high enough numbers to reelect Obama. A fair number of liberals fear the same thing. But there’s no analytic reason to believe this. The Obama campaign’s ground game seems to be as good as any in the business, and Obama voters are telling pollsters that they’re likely to vote in big enough numbers to give him the key swing states he needs to win. That’s the current state of our knowledge. It might be wrong, but if it is, the question isn’t going to be why Nate Silver went astray. The question is going to be, why was 2012 the year when people suddenly started lying to telephone pollsters?

UPDATE: Asawin Suebsaeng has a roundup of all the prognosticators here. It’s a nice, Cliff Notes version of who’s who and what they’re saying.
 

Thank you!

We didn't know what to expect when we told you we needed to raise $400,000 before our fiscal year closed on June 30, and we're thrilled to report that our incredible community of readers contributed some $415,000 to help us keep charging as hard as we can during this crazy year.

You just sent an incredible message: that quality journalism doesn't have to answer to advertisers, billionaires, or hedge funds; that newsrooms can eke out an existence thanks primarily to the generosity of its readers. That's so powerful. Especially during what's been called a "media extinction event" when those looking to make a profit from the news pull back, the Mother Jones community steps in.

The months and years ahead won't be easy. Far from it. But there's no one we'd rather face the big challenges with than you, our committed and passionate readers, and our team of fearless reporters who show up every day.

Thank you!

We didn't know what to expect when we told you we needed to raise $400,000 before our fiscal year closed on June 30, and we're thrilled to report that our incredible community of readers contributed some $415,000 to help us keep charging as hard as we can during this crazy year.

You just sent an incredible message: that quality journalism doesn't have to answer to advertisers, billionaires, or hedge funds; that newsrooms can eke out an existence thanks primarily to the generosity of its readers. That's so powerful. Especially during what's been called a "media extinction event" when those looking to make a profit from the news pull back, the Mother Jones community steps in.

The months and years ahead won't be easy. Far from it. But there's no one we'd rather face the big challenges with than you, our committed and passionate readers, and our team of fearless reporters who show up every day.

We Recommend

Latest

Sign up for our newsletters

Subscribe and we'll send Mother Jones straight to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate

We have a new comment system! We are now using Coral, from Vox Media, for comments on all new articles. We'd love your feedback.