Mask studies reach a new scientific low point

by Vinay Prasad

Masking is a divisive issue, however, at the end of the day, it is a scientific question. Pre-pandemic, community masking was discouraged because the pre-existing evidence was negative. This is why Fauci was critical of it in early March 2020 on 60 minutes. In the pandemic, masking has become a marker of politics. Good liberals wear them and bad conservatives don’t.
 
As a scientist stuck in the middle, with my colleagues J. Darrow and I. Liu, we performed an umbrella review of this topic. We found very poor quality data, insufficient to support community masking, particularly for years on end. Cloth masks had especially bad data. Data to support masking kids was absolutely absent. Worse however, was how little we learned during COVID19.
 
The CDC did not run a single randomized controlled trial (RCT). An individual RCT called DANMASK was negative, but this trial was powered for a 50% reduction in infection. Some thought that was asking too much (we will return to this)! A cluster RCT in Bangladesh was negative for cloth masks, and very modestly positive for surgical masks. However, further data revealed imbalances in the starting size— likely because the trial failed to achieve concealment, leading more people to sign up in the intervention arm (who may be less committed to report + Covid symptoms; biasing results). Furthermore, absolute event differences were v small. These facts, combined with prior literature, suggest that confidence even in surgical masks is extremely low. Meanwhile, there are no RCTs in kids, a catastrophic research failure.
 
Enter a new study in MMWR— the CDC’s pet journal; It is getting widely tweeted and cited, and that is unfortunate. The paper is entirely, irredeemably flawed. Its flaws are so evident that it should not have been published nor promoted. When an issue is deeply polarizing, publishing bad science helps no one. It cannot convince skeptics, proponents don’t need convincing, and it deepens mistrust in institutions. Let’s consider the paper.
 

 
The paper is a case control, test negative study. Basically, anyone in California btw Feb and Dec 2021 who got a COVID19 test could be enrolled. People were told if they tested + or -. Two days later, someone tried to call COVID-positive people. 13.4% of people who tested positive answered the phone. For each positive person, a negative person matched by age and sex who was called was identified. Only 8.9% of people who tested negative answered the phone.
 
We could stop right now. Very few people answered the phone. Moreover, there may be a big difference in who answered the phone. A person sick enough to be tested for COVID, who was +, might not be doing so well 2 days later. Which ones answered? How about those who tested negative? Are thee comparable people? Sadly, researchers continued….
 
Among people who answered the phone, people who said they had not spent time in indoor public settings were discarded. People who had a known COVID19 contact were discarded. The analysis just compares those who tested + vs – if they said they spent time in indoor public settings.
 
It turns out that people who tested positive were different than those who tested negative.
 

 
77% of people who tested positive got tested because of symptoms
 
17% who tested negative got tested because of symptoms
 
More people who tested negative, got tested “just cuz”
 

 
Ok, this was opportunity #2 to stop the voyage of the titanic. The reason for testing is very different. Very likely different groups of people are getting tested. People who are sick vs. people who are worried well/ people who work at companies that have been bamboozled by COLOR to offer asymptomatic testing. Sadly, the paper continued….
 
You know the rest of the story.
 
Among those who tested +, 9.3% never work mask in indoor settings and 93% wore it any time, and 60% wore it all the time. Among those who tested negative; 3% never wore, it 96% wore it any time, and 69% wore it always. Here it is
 

 
Do the math and people who tested + were less likely to wear masks than those who tested -.
 

 
The effect size is large. So large that DANMASK wasn’t underpowered after all! 😉 Can’t have it both ways, can you.
 
The paper did one more analysis, in a subset of people— though I cannot find just how many met this inclusion criteria by date (can anyone help me in comments?)- looking at type of mask.
 

 
They end with the now viral figure (no pun intended);
 
Of course, they forget to say cloth mask was NOT SIGNIFICANT p = .1.
 

 
What is wrong with this paper?

1. The people who test + and those who test – were seeking testing for totally different reasons. You are comparing sick people to people who wanted to get tested ‘just cuz’ or whose employer was swindled by these testing companies. Some people (more who test -) were getting tested before a medical procedure— these people may be extremely fastidious pre-procedure. I know I absolutely would not want to get sick before an elective surgery, and may even further alter my behavior. (though I am pretty chill)
2. In case you don’t see how this is a bias, the explanation is that the people getting tested who test neg. are far more affluent/ neurotic/ or precautious and will do anything they are told more than the avg. Joe. They definitely wear masks more. Even if masks did nothing, there would be an association. If you went on twitter for months and told them carrying a Fauci statue in their pocket would protect them from COVID, you could prove Fauci statues work with this design. Put more technically, the root virtue of test negative strategy (comparability) is violated and unmeasured confounding is injected. Paper cannot be salvaged.
3. The response rate is poor. I thought that awful day care study had a low response rate, but this one is lower. When you get a response rate this low you wonder if you are inserting biases you cannot even imagine. Who are the sick COVID people who answer the phone. Are they the least sick ones? The sick ones don’t take calls. Who are the people who answer the phone who test negative? The most anxious ones? The most gullible? ( I never answer unknown callers). Perhaps the ones most likely to carry Fauci statues in their pocket? (aka cloth masks)
4. It’s self reported. Mask usage is self reported. A person who was just told they have covid19 might be more likely to assume or believe they must have slipped in terms of mask usage. Self reporting after results are known is a huge bias. A true believer will convince themselves they didn’t wear masks as well as they ought to.

Read more