Tweet
With today's decision by the FDA Advisory Committee to recommend an EUA for the Pfizer / BioNTech COVID-19 vaccine formulation for 5 to 11-year-old children, we can assume that an EUA will follow.
I have seen it alleged on this forum that the recommendation was made despite a total lack of safety data. This is incorrect; however, it is important for caregivers to understand what is and isn't known about the vaccine's performance. Also, we may assume a similar hearing for Moderna is forthcoming, as their trials have been keeping pace.
We do not yet have a full reviewed publication of the trial results. This is coming -- we can expect something like the results for 12-15-year-olds to be published soon. In the meantime, we have a reasonably complete summary of the results as presented at the FDA hearing. Those materials can be found here.
Vaccine efficacy
Taken from page 21 of this package, the overall efficacy claimed is about 90%. In this case they appear to be using symptomatic efficacySafety Data
Given past results, there was little question about efficacy, but safety is a bigger concern -- and that drove the study design. We are thus far basing efficacy estimates on a dozen infections, but we have a few thousand subjects to give us some idea of side effects.
Safety data begins on Page 24. The reported reactions are surprisingly low -- again, probably a consequence of younger people's immune systems rebalancing faster. I know of only one person who didn't have a noticeable fever after a second shot of Pfizer, and certainly none after double Moderna, but that doesn't seem to be the case here. Might be the lower dosage, don't know.
The counts of severe events are really low. Across the board this isn't discernible from placebo. Not a surprise but good to see.
I'm intrigued by the comment on Page 30 about Serious Adverse Events, apparently including things like kids crashing their bicycles and getting bit by ticks -- this really demands some explanation in the full paper., as they claim none were related to vaccination. We're talking about a single digit count, though.
They also make a point to talk about myocarditis and pericarditis and note that they didn't find any, and this raises a question about the size of the trial. The estimate of vaccine-induced myocarditis is estimated at 105 per million -- approximately seven times the background rate in young adults -- so given our ensemble of 3,109 subjects, we'd expect to find only three. This trial suggests the rate of incidence isn't very high, but it doesn't give us much in the way of a confidence interval.
If they expected to find up to three, we may suppose the reviewers would have been comfortable with four or five -- or we're admitting that we don't actually know what decision threshold to use. There is a possibility that an unlucky wave of these would result in suspension of the license, like we saw with CVST in the Janssen vaccine.
I personally would like to see some better numbers here. I don't know whether the 5-11 group (at reduced dose) has a different response than 12-15 (at full power), and these numbers don't tell us. It suggests the rate is lower, but it could be artifactual. The problem is that this is expensive, and borderline unethical to grow the study pool until you cause a side-effect... so either we draw inferences from individual cases, or we should roll this out slowly until we come up with a valid rate.
Balance of Risks
Pfizer also prepared a comparison of relative risks, available here. This is kind of a funny document. There is nothing like complex modeling going on here -- instead, everything is linearized, making it more like a Maximum Expected Value / Worst Case Analysis estimate depending on the scenario.
The scenarios they're considering are all six-month blocks of time, partly to assume a steady state in vaccine efficacy. I can see a temptation to interpret this as a prelude to recommending a six-month booster schedule, but this doesn't appear to have any support. One way to look at this is to see whether there is a definite short-term return. Another is to simplify the estimate -- there is no consideration of reduced secondary transmission, for instance, mooting a possible point of contention, as vaccination will lead to fewer infections, but the error bars will be large.
I found these scenarios more useful in terms of clarifying the assumptions and numbers being used by the planners. For instance, there is the model for myocarditis outcomes. Another is the uncertainty in death rate -- described on Page 4, they note a factor of three difference between COVID NET and CDC Tracker, and this has to do with the sampling nature of COVID NET relying on a subset of "representative" hospitals and states. This uncertainty is to be expected given the statistically low percentage and probable dependence on hidden variables.
Anyway, the case being made here is not really a fair comparison, given the mortality signal from vaccination is infinitessimal in terms of the trial. Comparing hospitalization / ICU from COVID-19 vs. myocarditis is a bit more reasonable, but open to interpretation.
The variation in the scenarios illustrates that it all depends on what infection rate is realized. Many detractors will simply dismiss this result on the basis of a personal belief in negligible rates, and this is a weakness of the approach.
If I ran a cost-benefit study (something I swore to never do again, but that's another story
) I would instead try to bound exposure over the full duration of the pandemic. This is going to mean making some assumptions about erosion of protectivity, but it won't be so sensitive to instantaneous exposure rate, something that varies by two orders of magnitude. I would also look for other categories of benefits and risks, following papers such as this one -- it is true that child mortality from COVID-19 is quite low, but that doesn't really tell the whole story.
This study was limited to the few directly observable factors that came out of this particular trial. It was not instrumented for the purposes of a total risk balance, and as such the study is more of a qualitative assessment. That leaves it open to criticism, but it is still instructive.
Next Steps
I still want to see refereed publication of the results. I also think we may learn something once the controlled dose counts go from 5,000 to 500,000, so I would be a bit wary for now. However, and unsurprisingly, this all looks pretty good so far.
I have seen it alleged on this forum that the recommendation was made despite a total lack of safety data. This is incorrect; however, it is important for caregivers to understand what is and isn't known about the vaccine's performance. Also, we may assume a similar hearing for Moderna is forthcoming, as their trials have been keeping pace.
We do not yet have a full reviewed publication of the trial results. This is coming -- we can expect something like the results for 12-15-year-olds to be published soon. In the meantime, we have a reasonably complete summary of the results as presented at the FDA hearing. Those materials can be found here.
Vaccine efficacy
Taken from page 21 of this package, the overall efficacy claimed is about 90%. In this case they appear to be using symptomatic efficacySafety Data
Given past results, there was little question about efficacy, but safety is a bigger concern -- and that drove the study design. We are thus far basing efficacy estimates on a dozen infections, but we have a few thousand subjects to give us some idea of side effects.
Safety data begins on Page 24. The reported reactions are surprisingly low -- again, probably a consequence of younger people's immune systems rebalancing faster. I know of only one person who didn't have a noticeable fever after a second shot of Pfizer, and certainly none after double Moderna, but that doesn't seem to be the case here. Might be the lower dosage, don't know.
The counts of severe events are really low. Across the board this isn't discernible from placebo. Not a surprise but good to see.
I'm intrigued by the comment on Page 30 about Serious Adverse Events, apparently including things like kids crashing their bicycles and getting bit by ticks -- this really demands some explanation in the full paper., as they claim none were related to vaccination. We're talking about a single digit count, though.
They also make a point to talk about myocarditis and pericarditis and note that they didn't find any, and this raises a question about the size of the trial. The estimate of vaccine-induced myocarditis is estimated at 105 per million -- approximately seven times the background rate in young adults -- so given our ensemble of 3,109 subjects, we'd expect to find only three. This trial suggests the rate of incidence isn't very high, but it doesn't give us much in the way of a confidence interval.
If they expected to find up to three, we may suppose the reviewers would have been comfortable with four or five -- or we're admitting that we don't actually know what decision threshold to use. There is a possibility that an unlucky wave of these would result in suspension of the license, like we saw with CVST in the Janssen vaccine.
I personally would like to see some better numbers here. I don't know whether the 5-11 group (at reduced dose) has a different response than 12-15 (at full power), and these numbers don't tell us. It suggests the rate is lower, but it could be artifactual. The problem is that this is expensive, and borderline unethical to grow the study pool until you cause a side-effect... so either we draw inferences from individual cases, or we should roll this out slowly until we come up with a valid rate.
Balance of Risks
Pfizer also prepared a comparison of relative risks, available here. This is kind of a funny document. There is nothing like complex modeling going on here -- instead, everything is linearized, making it more like a Maximum Expected Value / Worst Case Analysis estimate depending on the scenario.
The scenarios they're considering are all six-month blocks of time, partly to assume a steady state in vaccine efficacy. I can see a temptation to interpret this as a prelude to recommending a six-month booster schedule, but this doesn't appear to have any support. One way to look at this is to see whether there is a definite short-term return. Another is to simplify the estimate -- there is no consideration of reduced secondary transmission, for instance, mooting a possible point of contention, as vaccination will lead to fewer infections, but the error bars will be large.
I found these scenarios more useful in terms of clarifying the assumptions and numbers being used by the planners. For instance, there is the model for myocarditis outcomes. Another is the uncertainty in death rate -- described on Page 4, they note a factor of three difference between COVID NET and CDC Tracker, and this has to do with the sampling nature of COVID NET relying on a subset of "representative" hospitals and states. This uncertainty is to be expected given the statistically low percentage and probable dependence on hidden variables.
Anyway, the case being made here is not really a fair comparison, given the mortality signal from vaccination is infinitessimal in terms of the trial. Comparing hospitalization / ICU from COVID-19 vs. myocarditis is a bit more reasonable, but open to interpretation.
The variation in the scenarios illustrates that it all depends on what infection rate is realized. Many detractors will simply dismiss this result on the basis of a personal belief in negligible rates, and this is a weakness of the approach.
If I ran a cost-benefit study (something I swore to never do again, but that's another story
) I would instead try to bound exposure over the full duration of the pandemic. This is going to mean making some assumptions about erosion of protectivity, but it won't be so sensitive to instantaneous exposure rate, something that varies by two orders of magnitude. I would also look for other categories of benefits and risks, following papers such as this one -- it is true that child mortality from COVID-19 is quite low, but that doesn't really tell the whole story. This study was limited to the few directly observable factors that came out of this particular trial. It was not instrumented for the purposes of a total risk balance, and as such the study is more of a qualitative assessment. That leaves it open to criticism, but it is still instructive.
Next Steps
I still want to see refereed publication of the results. I also think we may learn something once the controlled dose counts go from 5,000 to 500,000, so I would be a bit wary for now. However, and unsurprisingly, this all looks pretty good so far.
Comment