Do We Exaggerate Our Political Differences?

Facts matter: Sign up for the free Mother Jones Daily newsletter. Support our nonprofit reporting. Subscribe to our print magazine.


Andrew Gelman points me toward a new paper today that attempts to measure political polarization. The authors use NES survey data to measure (a) average positions from self-identified Democrats and Republicans on ten issues, and (b) perceptions of how big those differences are. They found that perceptions were consistently higher than reality: both Democrats and Republicans tended to overestimate how polarized we really are, and the more partisan you are the more you overestimate polarization. The full paper is here.

I’ve got a couple of problems with this paper, though. First, the dataset spans 30 years and ends in 2004. So even if the effect they find is real, it doesn’t tell us much about polarization, both real and imagined, over the past decade or so.

Second, and more important, is the effect real? There’s an obvious problem with how the authors calculate polarization:

We calculated perceived polarization, for every issue and respondent, as the respondent’s estimated attitude of the “Republican Party” minus the respondent’s estimated attitude of the “Democratic Party.” Higher numbers indicated greater polarization. We calculated actual polarization as the mean attitude of all respondents who self-identified as Republican minus the mean attitude of all respondents who self-identified as Democrat. We then calculated exaggerated polarization as perceived polarization minus actual polarization for each respondent on each issue in every year.

The problem is that when people are asked to estimate the actual positions of Democrats and Republicans, they probably aren’t thinking about national averages. More likely, they’re thinking about the positions of national leaders with big megaphones: presidents, governors, members of Congress, etc., all of whom are likely to have more strongly partisan positions. If that’s the case, perceptions of polarization might actually be fairly accurate. The authors are aware of this, so they ran a test to see if perceptions changed depending on how the questions were worded:

To directly address this possibility, we conducted a Web-based experiment to ascertain whether different ways of phrasing the attitude questions would produce different estimates of perceived polarization.

….Democratic or Republican targets were described using one of the four following prompts: (a) Where would you place the Democratic [Republican] Party on this scale? (b) Where would you place those people who identify as Democrats [Republicans] on this scale? (c) Where would you place Democratic [Republican] Party officials (that is, Democrats [Republicans] who hold an elected position at any level of government) on this scale? (d) Where would you place those people who identify as Strong Democrats [Republicans] on this scale?

The authors found no difference in perceptions based on question phrasing. Unfortunately, I don’t find this especially persuasive. Only about 70 people responded to each phrasing, so this is a pretty small sample set. Furthermore, I have my doubts that most people really have clear beliefs about these differences in the first place. For example, the authors report “somewhat surprisingly” that when the attitude question referred to party officials, perceived polarization was significantly lower. As they say, that doesn’t really make much sense, and it suggests that respondents simply weren’t thinking very hard about this. I suspect that happens a lot when you pay people a buck or two to fill out a questionnaire using Amazon’s Mechanical Turk.

So….I’m a little skeptical of this. It’s interesting in a limited way, but the differences between actual and perceived polarization were generally fairly modest; the data set is old; and it’s unclear that the authors’ definition of “actual” polarization corresponds to the real world. I don’t really doubt that the media exaggerates partisan differences these days, but I’d nonetheless take this study with a grain of salt.

THE TRUTH...

is the first thing despots go after. An unwavering commitment to it is probably what draws you to Mother Jones' journalism. And as we're seeing in the US and the world around, authoritarians seek to poison the discourse and the way we relate to each other because they can't stand people coming together around a shared sense of the truth—it's a huge threat to them.

Which is also a pretty great way to describe Mother Jones' mission: People coming together around the truth to hold power accountable.

And right now, we need to raise about $400,000 from our online readers over the next two months to hit our annual goal and make good on that mission. Read more about the information war we find ourselves in and how people-powered, independent reporting can and must rise to the challenge—and please support our team's truth-telling journalism with a donation if you can right now.

payment methods

THE TRUTH...

is the first thing despots go after. An unwavering commitment to it is probably what draws you to Mother Jones' journalism. And as we're seeing in the US and the world around, authoritarians seek to poison the discourse and the way we relate to each other because they can't stand people coming together around a shared sense of the truth—it's a huge threat to them.

Which is also a pretty great way to describe Mother Jones' mission: People coming together around the truth to hold power accountable.

And right now, we need to raise about $400,000 from our online readers over the next two months to hit our annual goal and make good on that mission. Read more about the information war we find ourselves in and how people-powered, independent reporting can and must rise to the challenge—and please support our team's truth-telling journalism with a donation if you can right now.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate