Fight disinformation: Sign up for the free Mother Jones Daily newsletter and follow the news that matters.

Back during World War II, the RAF lost a lot of planes to German anti-aircraft fire. So they decided to armor them up. But where to put the armor? The obvious answer was to look at planes that returned from missions, count up all the bullet holes in various places, and then put extra armor in the areas that attracted the most fire.

Obvious but wrong. As Hungarian-born mathematician Abraham Wald explained at the time, if a plane makes it back safely even though it has, say, a bunch of bullet holes in its wings, it means that bullet holes in the wings aren’t very dangerous. What you really want to do is armor up the areas that, on average, don’t have any bullet holes. Why? Because planes with bullet holes in those places never made it back. That’s why you don’t see any bullet holes there on the ones that do return. Clever!

On a related note — related because it involves both an air force and some counterintuitive statistical reasoning — Henry Farrell writes about a passage from Justin Fox’s Myth of the Rational Market involving the great behavioral economist Daniel Kahneman. Here is Kahneman reacting to an Israeli Air Force flight instructor who told him that whenever he chewed out a student he got better performance the next time out, but whenever he praised a student he got worse performance:

As a man trained in statistics, Kahneman saw that of course a student who had just brilliantly executed a maneuver (and was thus praised for it) was less likely to perform better the next time around than a student who had just screwed up. Abnormally good or bad performance is just that — abnormal, which means it is unlikely to be immediately repeated. But Kahneman could also see how the instructor had come to his conclusion that punishment worked. “Because we tend to reward others when they do well and punish them when they do badly, and because there is regression to the mean,” he later lamented, “it is part of the human condition that we are statistically punished for rewarding others and rewarded for punishing them.”

Likewise, we tend to go to the doctor only when an illness has become really bad. Sometimes, though, that just means the illness is at its lowest point and has nowhere to go but up. And sure enough, after we see the doctor we get better. But it’s not always because of something the doctor did. Sometimes it’s just a natural consequence of when humans choose to see them.

Finally, on yet another related note — related this time solely because it involves counterintuitive statistical reasoning, not because it involves an air force — check out Nate Silver’s brief seminar on how weak most of us are at calculating how much variance there is in the things we do every day. There’s a lot of natural variance in everyday life, and there’s not always a reason for it. But we like to make up reasons anyway. Statistics is a profoundly unnatural enterprise for most of us.

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with the Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with the Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate