We Have No Idea Which Interventions Work Against COVID-19

Handwashing definitely works against COVID-19. But does anything else?Kevin Drum

Fight disinformation: Sign up for the free Mother Jones Daily newsletter and follow the news that matters.

A while back I mentioned that we were in new territory with COVID-19: we’ve had similar or bigger pandemics before, but we’ve never had one where we adopted widespread countermeasures and then studied how well they worked. This was an offhand comment and I got curious today if it was really true. Spoiler alert: It is.

I went looking for empirical studies done before 2020 since I was interested only in pandemics prior to COVID-19. It didn’t take long to come across “Effectiveness of workplace social distancing measures in reducing influenza transmission: a systematic review,” which was published in 2018. It’s a review of previous studies of social distancing and other countermeasures in “socially dense community settings, such as schools or workplaces,” and it’s far more comprehensive than anything I could do myself. Here’s what they found:

  • A grand total of 15 studies.
  • Of which, only three were epidemiological (the others were modeling studies).
  • Of which, the overall risk of bias was rated “critical” in one, meaning the “study is too problematic to provide any useful evidence on the effects of intervention.”
  • That leaves two studies. The risk of bias was rated “serious” in both, but at least they weren’t completely useless.
  • Of those, one was a study of ordinary seasonal influenza in 2007-08 and looked only at employees who could work at home vs. those who couldn’t.

The sole remaining study was conducted on a group of 1,015 miltary personnel in Singapore during the 2009 H1N1 pandemic. It compared a control group (no intervention) to a “normal” group (individuals were provided general health education on respiratory and hand hygiene and were advised to seek medical care if ill) and an “essential” group (which received enhanced surveillance with isolation, segregation, and personal protective equipment). The study found that 44 percent of the control group contracted the flu compared to 17 percent of the normal group and 11 percent of the essential group. It also found that the pandemic peaked earlier in normal units compared to the control group.

So that’s it: a single study that compared groups to each other but didn’t try to establish the efficacy of individual interventions. Normal hygeine advice apparently had a big effect, reducing incidence from 44 percent to 17 percent, but only the “essential” group practiced any kind of social distancing, and it reduced incidence of the flu only from 17 percent to 11 percent. This may sound worthwhile even though it’s small, but don’t forget that this study had “serious” problems with possible bias.

Effectively, this means we’re flying blind. We have loads of modeling estimates of various interventions:

  • Quarantines
  • Stay-at-home orders
  • School closings
  • Social distancing
  • Mask wearing
  • Bans on large gatherings
  • Closure of restaurants
  • Closure of non-essential businesses
  • Mass testing and contact tracing

For practical purposes, however, we have no reliable empirical data at all on any of these measures. It makes sense that some or all of them have an effect—and it’s probably safe to say that all of them put together have an effect—but we have no idea which particular ones have a large effect vs. which ones have a small effect. And given the vast range of assumptions used in various models, it’s not clear to me that we can trust models to tell us anything useful at the level of specific interventions.

It’s astonishing how much we’re learning about COVID-19 on literally a daily basis. In some ways it works like any other pandemic, but in other ways it truly appears to be unique. Given this, and given the complete lack of good empirical studies of past pandemics, we should be very, very cautious about insisting on any particular intervention as either critical or dispensable. We just don’t know.

LET’S TALK ABOUT OPTIMISM FOR A CHANGE

Democracy and journalism are in crisis mode—and have been for a while. So how about doing something different?

Mother Jones did. We just merged with the Center for Investigative Reporting, bringing the radio show Reveal, the documentary film team CIR Studios, and Mother Jones together as one bigger, bolder investigative journalism nonprofit.

And this is the first time we’re asking you to support the new organization we’re building. In “Less Dreading, More Doing,” we lay it all out for you: why we merged, how we’re stronger together, why we’re optimistic about the work ahead, and why we need to raise the First $500,000 in online donations by June 22.

It won’t be easy. There are many exciting new things to share with you, but spoiler: Wiggle room in our budget is not among them. We can’t afford missing these goals. We need this to be a big one. Falling flat would be utterly devastating right now.

A First $500,000 donation of $500, $50, or $5 would mean the world to us—a signal that you believe in the power of independent investigative reporting like we do. And whether you can pitch in or not, we have a free Strengthen Journalism sticker for you so you can help us spread the word and make the most of this huge moment.

payment methods

LET’S TALK ABOUT OPTIMISM FOR A CHANGE

Democracy and journalism are in crisis mode—and have been for a while. So how about doing something different?

Mother Jones did. We just merged with the Center for Investigative Reporting, bringing the radio show Reveal, the documentary film team CIR Studios, and Mother Jones together as one bigger, bolder investigative journalism nonprofit.

And this is the first time we’re asking you to support the new organization we’re building. In “Less Dreading, More Doing,” we lay it all out for you: why we merged, how we’re stronger together, why we’re optimistic about the work ahead, and why we need to raise the First $500,000 in online donations by June 22.

It won’t be easy. There are many exciting new things to share with you, but spoiler: Wiggle room in our budget is not among them. We can’t afford missing these goals. We need this to be a big one. Falling flat would be utterly devastating right now.

A First $500,000 donation of $500, $50, or $5 would mean the world to us—a signal that you believe in the power of independent investigative reporting like we do. And whether you can pitch in or not, we have a free Strengthen Journalism sticker for you so you can help us spread the word and make the most of this huge moment.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate