COVID Transmissions for 9-20-2021

FDA panel recommends booster doses only in certain groups; CDC shows mRNA vaccines more effective

Sep 20, 2021

Greetings from an undisclosed location in my apartment. Welcome to COVID Transmissions.

It has been 670 days since the first documented human case of COVID-19. In 670, King Oswiu of Northumbria died on a pilgrimage to Rome. The same journey is a 3 hour flight, nowadays—but you’d have to contend with the risk of COVID-19 while traveling.

Today we’ll talk about the FDA advisory committee recommendation on booster doses, but also I share some data comparing how good the various vaccines are at keeping you out of the hospital. They’re all very good! Some appear to be a bit better than others.

Schedule update, first: We are almost at the end of the Jewish holiday season. The festival of Sukkot is now coming up, when we build little shacks to “dwell” in (more on this later). Consequently, this holiday will affect the newsletter schedule somewhat. For the next two weeks, I won’t be writing issues on Wednesday and will adjust the week’s second issue according to make sure you’re not waiting a long time between issues. Thus, COVID Transmissions will come out on Mondays and Thursdays.

They say the most important thing in online communication is consistency, so please rest assured that once the holidays are behind us I plan to be back on a regular Monday-Wednesday-Friday schedule.

Bolded terms are linked to the running newsletter glossary.

Keep COVID Transmissions growing by sharing it! Share the newsletter, not the virus. I love talking about science and explaining important concepts in human health, but I rely on all of you to grow the audience for this, which you can do by using this button here:

Share COVID Transmissions

Now, let’s talk COVID.

FDA panel issues decision on boosters

The FDA advisory committee covering COVID-19 vaccines issued a decision last week recommending the use of boosters for Pfizer’s COVID-19 vaccine (“Comirnaty”) for patients who are 65 and over or who are otherwise in vulnerable demographics: https://www.cnbc.com/2021/09/17/fda-panel-begins-voting-on-pfizers-covid-booster-doses-rejecting-shots-for-general-public.html

That recommendation was made unanimously, 18-0.

This came after much conversation over whether third doses would be recommended for the general population. That proposal was defeated 16-2.

These decisions are in line with my scientific understanding of the situation that we are currently in. There has been some evidence of immune waning in the people who got vaccinated at the very beginning of the vaccine rollout. There are two ways to interpret that information. First, we might presume that the vaccine itself begins to wane at after about 8 months and so a booster is needed 8 months out. Alternatively, we could consider that older people were typically prioritized at the front of the line, and older people also generally have more problems maintaining robust immune memory. One of these two things is true here, but no matter the interpretation, mostly the question of third doses right now applies to those people who got the vaccine the earliest—in this case, older folks and people with vulnerable conditions.

In the future, we may learn more about immune memory with COVID-19 vaccines and that may lead to a change in the recommendation, but based on what we know right now there is good reason to recommend a third dose in the most vulnerable populations, and there isn’t good reason to recommend it in other populations. With medicine, it’s better not to give the patient something unless you have a good reason for it, so they recommended it for the people in whom there is a good reason. That’s really the whole story here—and you’ve heard my thoughts on general-population boosters at length in previous issues, so let’s just leave it there. Right now, this recommendation is one I agree with and I’m glad to see the expert panel doing its job.

CDC compares vaccines; mRNA vaccines clearly more effective at preventing hospitalization

The CDC, writing in Morbidity and Mortality Weekly Report (MMWR), announced an analysis of relative vaccine effectiveness for the Pfizer, Moderna, and Janssen (J&J) vaccines against hospitalization: https://www.cdc.gov/mmwr/volumes/70/wr/mm7038e1.htm

I quite like the approach here, because the study was looking at a concrete endpoint that is not ambiguous. If the study had been looking at infection overall, we might have an issue because detecting infection is not a reliable process. Many infections go undetected because they cause no symptoms, and others are only detected because someone happened to need a test for school or work. Infections that cause no symptoms, in a vaccinated person, are an example of the vaccine working. The point of the vaccine is to prevent disease. So I don’t like studies that hold to a “no infections” standard of protection.

Meanwhile, hospitalization is much less subject to variation. People go to the hospital when they are seriously ill. They are hospitalized when there is good reason to admit them. There is not some random chance that an asymptomatic person will be hospitalized—that is a little silly to even contemplate. So we can expect this type of endpoint to be a lot more reliable to observe.

This having been said, the results were as follows (presenting vaccine efficacy with the 95% confidence interval):1

Moderna: 93% (91-95%)
Pfizer: 88% (85-91%)
J&J: 71% (56-81%)

I’m going to get very statistical in a moment, so for those of you looking for the bottom line as I see it: I think it’s clear from this that the mRNA vaccines, given in two doses, are more effective at preventing hospitalization than the single-dose J&J vaccine. However, I do not think the difference between the Moderna and Pfizer vaccines here is very meaningful, and I’ll explain why.

The reason I gave the confidence intervals is to demonstrate that the range of possible vaccine effectiveness estimates for the Moderna and Pfizer vaccines overlaps. In clinical science—in any science, really—we know that our experimentally observed numbers contain some amount of error. All individual numbers are just an estimate. A “confidence interval” represents the range of values where we’re 95% sure that the “true” number really is. When confidence intervals overlap, there’s a chance that both estimated values are a little off, and the “true” values for both estimates are, actually, the same.

Confidence intervals are not the whole story, though. The study says that the difference between these two vaccines has a “p-value” of 0.011, which is below the 0.05 threshold that would normally be used to say this difference is “significant”, but I am unwilling to conclude that Moderna is meaningfully better than the Pfizer vaccine based on this. P-values represent the chance that the data appear different only be random variation. This p-value suggests that there is only a 1.1% chance that this is random variation, strictly speaking, but that is misleading. The thing about p-values is that they themselves have a certain chance of just being wrong. It is possible to game the statistics—unintentionally or intentionally—such that a “significant” p-value is obtained when it might not be otherwise.

One classic situation where p-values can be misleading is when relatively arbitrary study length conditions are used. Who’s to say that 5 months (the length of time that this study covered) is a better study length than 6 months or 8 months? We already suspect that effectiveness of these vaccines may wane after 8 months, so is 5 months really enough? Normally, in randomized clinical trials, we avoid this problem by not using a fixed follow-up to do our analyses. Instead, we look at a large population in advance, define an event of interest, randomize the patients to different treatment groups, and then run the study until a prespecified number of events of interest have happened in all groups in total.

Why is this important? Well, it’s the difference between my saying “let’s decide who gets the last chicken wing by flipping a coin” and then saying “best of 3?” after I lose the first toss vs my saying “let’s decide who gets the last chicken wing by the winner of the best of 3 coin flips”—before we’ve done any flipping. If you are looking at the events after the fact, consciously or unconsciously you may introduce bias by selecting a different follow-up period.

To my mind, this makes retrospective studies of real-world data a little more suspect when it comes to the concept of “statistical significance.” This is not a controlled study, so I don’t make a lot out of the p-value for Moderna vs Pfizer when the 95% confidence intervals overlap. In my opinion, the mRNA vaccines are probably about the same for the purposes of this endpoint. Even if the apparent edge that Moderna has is real, the effect is extremely small.

When it comes to the J&J vaccine, it’s a different story. Both mRNA vaccines were significantly different from the J&J vaccine with a p-value of less than 0.001. In other words, there is a 99.9% chance the difference between J&J and the mRNA vaccines here is a real effect. What’s more, the 95% confidence intervals for these vaccine estimates are not even close to overlapping. I feel…confident…that the mRNA vaccines are more effective in preventing hospitalization than the J&J vaccine, although the J&J vaccine’s effects are still incredible—particularly considering it is only one dose while the others have a two-dose schedule.

I’m not sure whether the difference here is due to the dosing schedule or something else, but I really do lean towards thinking the dosing schedule is the main reason.

Either way, what we have here is three vaccines that are very good at keeping people out of the hospital, and we have good data to prove it. If you’re still unvaccinated and can get your pick of these vaccines, perhaps these data will help you choose one. If you have already been vaccinated and now are questioning your choice—don’t. Your risk of hospitalization, with ANY of these vaccines, has fallen to somewhere between one third and one twentieth of what it was before you got vaccinated. That’s a huge benefit, especially considering there was no protection at all before these vaccines became available.

What am I doing to cope with the pandemic? This:

Building a shack

The Jewish holiday of Sukkot is coming, so over the weekend I helped my synagogue construct our sukkah. A sukkah (“סוכה” in Hebrew, plural sukkot) is, essentially, a shack. The holiday is rooted in the wanderings of the Children of Israel through the wilderness after leaving Egypt, a journey that involved living in sukkot for some period of time. We celebrate the holiday of Sukkot in remembrance of that, and we also construct sukkot to “dwell” in during this time. I put “dwell” in quotation marks because very few people actually live inside the things. More often, they eat meals in them as a minimum level of habitation. For city folk like myself, very few people construct their own sukkah (you aren’t allowed to make one indoors, among other rules, so pillow forts are decidedly not an option). Instead, we rely on communally-constructed sukkot that synagogues and other Jewish groups construct.

Building one of those is how I spent my Sunday!

Reader “Just Another Bozo on the Bus” sent another question regarding arguments with their friend about the effectiveness of ivermectin, this time bringing up something that I’ve heard too—claims that ivermectin had some sort of amazing effect in the Indian state of Uttar-Pradesh. I feel your pain, reader—I’ve heard this before too. Typically, in my experience, the people bringing this up don’t live in or anywhere near Uttar-Pradesh, but expect that somehow what they believe happened there will have global implications. Unfortunately, I’ve found they’re generally wrong both about what actually happened in Uttar-Pradesh as well as what it might mean for COVID-19 treatment. Here’s the reader comment:

Hi John - Happy Holidays!
My wingnut friend is now pointing out the "success" of ivermectin in India with articles like this: https://indianexpress.com/article/cities/lucknow/uttar-pradesh-government-says-ivermectin-helped-to-keep-deaths-low-7311786/ Are there any good studies of the Uttar Pradesh/India covid response and the results?

And here’s my reply:

I haven't seen any specific takedowns of the Uttar-Pradesh data, and I believe this is because almost nobody thinks the data out of India are accurate. India does not have effective healthcare surveillance for COVID-19 and the epidemiology there is widely thought to be inaccurate. Official numbers in India for the most recent surge were 400,000 deaths. Estimates based on excess deaths suggest 4,000,000 is closer to the true number. See here: https://www.cgdev.org/publication/three-new-estimates-indias-all-cause-excess-mortality-during-covid-19-pandemic
Even if the data are more reliable than they appear, the public figures don't suggest anything particularly out of the ordinary happened in Uttar Pradesh with regard to death rates. COVID-19 case fatality rates have been rather reliably 1% on average in a variety of places, though of course in some places they are higher and in some places they are lower. In Uttar Pradesh, they have been modestly higher over the course of the pandemic, with 1.71 million cases translating to just over 22,000 deaths in public numbers. Perhaps these are off by an order of magnitude and indeed it's 17 million cases with over 220,000 deaths, but whichever idea is correct, it's still a relationship that shows slightly more than 1% case-fatality rate. That does not strike me as a "success."
And finally, even if there were some kind of unusually low death pattern here—which there doesn't appear to be at all—correlation isn't causation. There are randomized clinical studies of ivermectin in COVID-19 which are ongoing. Those are where we get causal data from, not from speculative claims made by government officials based on public numbers that are widely believed to be inaccurate.
For some further background reading, I think that this article from Discover (edit: Cosmos, actually, but I wrote “Discover” accidentally in my comment back to this reader) Magazine is actually pretty fantastic as a summary of how to respond to this: https://cosmosmagazine.com/health/covid/what-we-know-about-ivermectin-correlation-is-not-causation/
The article sites the person that I've listened to most about this, epidemiologist Gideon Meyerowitz-Katz, who has blogged quite a lot about ivermectin and its apparent uselessness against COVID-19:
https://gidmk.medium.com/
Hopefully this all helps.

You might have some questions or comments! Send them in. As several folks have figured out, you can also email me if you have a comment that you don’t want to share with the whole group.

Join the conversation, and what you say will impact what I talk about in the next issue.

Also, let me know any other thoughts you might have about the newsletter. I’d like to make sure you’re getting what you want out of this.

Viral Transmissions

Discussion about this post