I do not think this study means what this writer thinks it means

(Added Sunday: another take on the same study from a writer at Vox.)

NYT, Bret Stephens, 21 Feb 2023: The Mask Mandates Did Nothing. Will Any Lessons Be Learned?

A conservative columnist for the NY Times claims a meta-study on the efficacy of masks for reducing the spread of respiratory illness has concluded that they didn’t work.

Or don’t work. I confess that when I first noticed this article, I thought it addressed mask mandates during the Covid pandemic. And my initial response was that every new pandemic could be different, and if masks didn’t work for Covid, they might well work for the next virus that comes along. The article struck me as a conservative finding a reason to resist, and resent, mask mandates of any type, in the past and in the future.

But apparently the meta-study considered 78 randomized controlled trials, only 6 of which were during Covid. And further, that “hand hygiene, physical distancing or air filtration” had no effect either.

Hard to believe. For respiratory illnesses, precautions against coming into contact with a virus have no effect?

Stephens links the study he’s discussing, called Physical interventions to interrupt or reduce the spread of respiratory viruses, but he doesn’t quote from it. He quotes from an interview with one of the study authors, Tom Jefferson, by journalist (?) Maryanne Demasi on her Substack site, where Jefferson rants about the media, the pandemic’s “overnight experts,” and even the publisher of his study.

Immediately you see they’re spinning and oversimplifying the truth, as if they have an axe to grind.

JEFFERSON: Yes, well, governments completely failed to do the right thing and demand better evidence. At the beginning of the pandemic, there were some voices who said masks did not work and then suddenly the narrative changed.

DEMASI: That is true, Fauci went on 60 minutes and said that masks are not necessary and then weeks later he changed his tune.

JEFFERSON: Same with New Zealand’s Chief Medical Officer. One minute he is saying masks don’t work, and the next minute, he flipped.

Not the way I recall it. Critics are always accusing scientists of flip-flopping, of changing their minds, when what they’re doing is change their responses to a changing situation. At the beginning of the pandemic there was a limited supply of masks, and people like Fauci discouraged people from wearing them in favor of letting medical personnel use them. As the supply increased, and the threat to the public of the virus increased (as more and more cases arose), then it became appropriate to advise the public to wear masks. Is this so difficult to understand?

\

Typically for scientific papers, the conclusions of the meta-study are couched in in qualifications, e.g. “The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions.” The very lengthy paper goes on with many details and many more qualifications.

This doesn’t stop Stephens.

Mask mandates were a bust. Those skeptics who were furiously mocked as cranks and occasionally censored as “misinformers” for opposing mandates were right. The mainstream experts and pundits who supported mandates were wrong.

No they weren’t. For one thing, the results of this study were not known at the time. This is called hindsight bias.

And Stephens admits, “What’s more, the analysis does not prove that proper masks, properly worn, had no benefit at an individual level.”

And he admits that Americans aren’t disciplined enough to wear masks properly.

Yet there was never a chance that mask mandates in the United States would get anywhere close to 100 percent compliance or that people would or could wear masks in a way that would meaningfully reduce transmission. Part of the reason is specific to American habits and culture, part of it to constitutional limits on government power, part of it to human nature, part of it to competing social and economic necessities, part of it to the evolution of the virus itself.

Stephens is seeing what he wants to see. And if masks truly don’t work to stop respiratory infections, should surgeons and nurses throw their masks away too? He concludes,

There’s a final lesson. The last justification for masks is that, even if they proved to be ineffective, they seemed like a relatively low-cost, intuitively effective way of doing something against the virus in the early days of the pandemic. But “do something” is not science, and it shouldn’t have been public policy. And the people who had the courage to say as much deserved to be listened to, not treated with contempt. They may not ever get the apology they deserve, but vindication ought to be enough.

Again, the people resisting the mandates were doing so out of resentment of government mandates, not because they had evidence or reason to think masks, social distancing, and hand hygiene wouldn’t work! (Remember a large proportion of the resistors thought the virus was a hoax, or a deliberate conspiracy; they were not reasonable people.) Maybe the measures would have worked had not the resistors ignored all that advice and undermined health advice that affected everyone.

And when circumstances of a new pandemic are mostly unknown, it’s entirely appropriate to do something reasonable, until evidence says otherwise. Is this study evidence otherwise? No, there are too many qualifications to the study’s conclusions, and too many reasons why people like Stephens see what they want to see. Remember, conservatives don’t do nuance; they think in black and white.

\\

Vox, Kelsey Piper, 22 Feb 2023: The new scientific review on masks and Covid isn’t what you think, subtitled “A meta-analysis seeks to be the last word on the effectiveness of masks, but finding answers in science isn’t that easy.”

A somewhat more balanced take.

For medical questions, a major source of high-quality, reputable meta-analyses is Cochrane, a UK-based nonprofit that publishes long and comprehensive meta-analyses of current evidence on medical and therapeutic interventions. … Cochrane reviews are frequently called the “gold standard” for evidence-based medicine.

But there are a lot of methodological decisions that go into a meta-analysis, and Cochrane brings its own set of assumptions to that table: they tend to exclude many studies as low-quality, and tend to be conservative in declaring that an effect exists.

The writer here summarizes the new meta-analysis. Then,

I think Jefferson — an Oxford University epidemiologist who has a number of eccentric and flatly nonsensical opinions about Covid-19, including that it didn’t originate in China and may have been circulating in Europe for years before its global emergence — is overstating his case. There is something we can learn from the Cochrane paper, but it’s as much about the process of science as it is about the effectiveness of masks.

Included among her reasons are that only 6 of the 78 studies were conducting during the Covid-19 pandemic; the study thus draws broad conclusions from studies about different kinds of viruses.

Also, the study doesn’t consider whether people actually wore masks, only whether they were told to wear them.

Studies that looked at Covid specifically found substantial evidence of mask efficacy, e.g. 35% reduction among individual 60 or older.

So why did the meta-analysis conclude that mask wearing made little or no difference?

Because their meta-analysis mixes these studies with many more pieces of research that were conducted before Covid-19 and found little effect of masks on the transmission of other illnesses like influenza.

And.

“I was really surprised that the Cochrane group let this go through,” Jake Eaton, a public policy and global health researcher who was the lead researcher on a Cochrane review of childhood nutrition, told me. “The fact it’s looking at masks across different settings and with different diseases makes it really tricky. Cochrane reviews are very good if you really want to assemble the most rigorous evidence and say, ‘Do we have a conclusive signal that this works?’ This is something of a perverse use of a Cochrane review.”

Piper concludes,

And I do think the government bears some responsibility for poor communication about masks: this is an intervention where evidence is limited and suggestive, not a sure thing. But I’m also frustrated by the way this quite bad meta-analysis has been seized on as proof that masks don’t work.