<a href="http://en.wikipedia.org/wiki/File:Paris_Tuileries_Garden_Facepalm_statue.jpg">Alex E. Proimos</a>/Wikimedia Commons
The United Nations’ Intergovernmental Panel on Climate Change is a big, big production. Its reports, released roughly every five years, are considered the gold standard of climate science, and are always a major media event. Thousands of scientists contribute to the reports, all of them volunteering their expertise to make the world just a little bit better.
There’s just one problem: According to a new paper out in Nature: Climate Change, the IPCC may be dramatically undermining its own work through one of its trademark tools: A system of language that the group uses to describe how certain (or uncertain) researchers are about its scientific findings. According to the new study, this system (which involves describing conclusions as “likely,” “very likely,” and so on) has the unfortunate effect of making people less sure than they ought to be of the IPCC’s most important conclusions.
Unintentionally, then, the IPCC seems to be doing just what climate skeptics and deniers are so often accused of: Sowing doubt.
The new study, by psychologist David Budescu of Fordham University and his colleagues, is actually the latest in a string of papers by these researchers showing that people systematically misunderstand what the IPCC means when it uses phrases such as “likely” and “very likely” to describe the strength of its conclusions. Take, for instance, the IPCC’s famous finding, in 2007, that “most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic [human-produced] greenhouse gas concentrations.” According to Budescu’s research, while the IPCC intends for “very likely” to mean a greater than 90 percent likelihood, that’s not necessarily the message the average person hears. Instead, when Budesco and his colleagues asked members of the public to assign a probability to the term “very likely,” the mean estimate people gave was just 62 percent.
In general, Budescu finds that when the IPCC assigns a high level of certainty to a conclusion, using terms like “very likely” or “extremely likely,” people adjust their interpretation downward, taking the conclusion to be considerably less certain than it actually is. When the IPCC assigns a low probability, meanwhile, people adjust their assessment upwards, taking the conclusion to be considerably more certain than it actually is. Thus for instance, when the IPCC calls a conclusion “very unlikely,” it means there is less than a 10 percent chance that it’s true. But the mean estimate given by members of the public for what this term means is 41 percent. To see how much confusion this can cause, just consider another 2007 IPCC statement: “It is very unlikely that climate changes of at least the seven centuries prior to 1950 were due to variability generated within the climate system alone.”
Budescu and his colleagues have found these results consistently, across samples. In 2009, they found as much with a sample of college students and members of a single university community. In 2012, they did so again with a nationally representative sample of Americans. And in the new Nature: Climate Change study, they present the same finding with citizens of 25 countries, having now conducted the research in multiple languages. Again and again, it would seem that the IPCC’s language about uncertainty backfires, and undermines itself. It sows doubt in the minds of the public.
Moreover, given that Budescu’s first paper on this subject was published in 2009, the IPCC should presumably know by now that its practices appear to have caused the public to be far more doubtful than it should be about the science of climate change. In fairness, the current approach exists for a reason: It avoids requiring scientists to be too precise about their level of certainty, and it allows for the possibility that different scientists would come up with somewhat different numbers for their extent of certainty.
“I think that they are finding their way slowly, and they are trying things,” remarks Budescu of the IPCC’s uncertainty practices. “A lot of the things that they are trying make sense, and are reasonable. I think they are slow in adjusting.”
Solving this problem would be quite simple: Budescu’s research shows that people’s misunderstanding of the IPCC’s language about uncertainty decreases if you simply include a numeric value next to the standard uncertainty language. Thus, instead of merely saying “very likely,” the IPCC could just prominently add “(> 90% likelihood),” or something similar. As it is, these numerical values are included as a footnote in the IPCC’s widely read “Summary for Policymakers” reports, and a box in the much less widely read technical report; Budescu’s research suggests they should appear throughout the text.
It is critical to underscore just how problematic the IPCC’s ill-calibrated uncertainty language is. The IPCC produces many thousands of words in its reports, and spends five or more years doing so; and yet generally, there is one sentence from each report that is almost universally quoted in the press, on blogs, and beyond. It is always the sentence that describes how certain the IPCC is about the conclusion that humans are causing global warming; and that sentence always contains the IPCC’s confusing uncertainty-speak. In 2001, the IPCC found the conclusion “likely”; in 2007, “very likely”; and in 2013, “extremely likely.” To the IPCC, that meant “greater than 66 percent likelihood,” “greater than 90 percent,” and “greater than 95 percent,” respectively. Based on the latest research, the public took away a very different message indeed.
Journalists may partly mitigate this problem, to be sure. Outlets like the New York Times and the Washington Post, in their coverage of last year’s IPCC report, took it upon themselves to include a numerical probability value as they explained the IPCC’s conclusion that it is now “extremely likely” that humans are driving global warming. Yet not all media outlets did: Take this report from ABC News, for instance; it included a clip of an IPCC official saying it’s “extremely likely” that humans are causing climate change but did not include any numerical explanation of what that means.
The IPCC has been extensively faulted in the past for a wide range of communications failures. Not all of them have easy fixes, but this one surely does.
