Statistical Considerations for Evaluating COVID Vaccine Protection

I refer to a recent Morbidity and Mortality Weekly Report (MMWR) to highlight a deficit in the reporting of statistical data: The failure to address the implications of the 95% confidence interval (95% CI).

The report pertains to estimates of COVID-19 vaccine effectiveness (VE) against emergency department or urgent clinic encounters, and hospitalizations among adults during SARS-CoV-2 (Delta) variant predominance. Regarding VE, the 95% CI is a measure of uncertainty associated with the level of VE obtained from the actual sample studied (the point estimate). Of course, other levels of confidence can be used for interval estimates, but 95% is typical in public health and medicine.

The 95% CI is used for generalizing or making an inference about the population, and can be pragmatically defined as a range of values around the point estimate that has a 95% probability of including the true level of VE for the population. The stricter and more theoretical definition and interpretation of confidence interval is based on “the hypothetical notion of considering the results that would be obtained if the study were repeated many times.” The size of the sample or subgroup is a key determinant of the precision of the 95% CI — smaller samples result in wider confidence intervals, which are less precise.

Best practice in statistical analysis and reporting regarding vaccine efficacy and effectiveness requires that both the 95% CI and the point estimate be fully interpreted and explained. However, it is not uncommon for researchers to merely mention the 95% CI in the methodology or results section of a report without addressing its implications for decision-making, while emphasizing the point estimate. This is a grave disservice to all.

Regarding the MMWR in question, while the point estimates of VE (mRNA) are very high, the lower limit of the 95% CI plausibly indicates that a lower level of VE can be expected if repeated studies were conducted on this population, and this is largely attributable to variation in research contexts, including personal characteristics. If this is not effectively communicated to the public, should the level of VE decrease in subsequent studies, the public may incorrectly interpret this as vaccine failure, when in fact, the lower level of VE may be accounted for by the 95% confidence intervals from previous studies.

Indeed, the MMWR supports that VE during the Delta wave “was significantly higher among Moderna vaccine recipients (95%) than among Pfizer-BioNTech (80%) or Janssen (60%) vaccine recipients.” Specifically, regarding COVID-19 hospitalization, the report notes the following VE point estimates:

  • mRNA-1273 (Moderna): 95% (95% CI 92-97)
  • BNT162b2 (Pfizer-BioNTech): 80% (95% CI 73-85)
  • Ad26.COV2.S (Janssen/Johnson & Johnson): 60% (95% CI 31-77)

The lower limit of the 95% CI indicates that the true value of the level of VE can be as low as 92% for Moderna, 73% for Pfizer-BioNTech, and 31% for Janssen, which albeit relatively low, marginally achieves the lower bound of “>30%” generally required by the FDA to consider a vaccine worthy of being widely deployed. Attention to the lower limit of the 95% CI is a conservative approach to assessing VE. While the Moderna and Pfizer-BioNTech mRNA vaccines demonstrated high levels of protection against moderate and severe COVID-19 resulting in hospitalization, the Moderna vaccine outperformed the other two vaccines, and the difference was statistically and substantively significant. This trend was reinforced in a subsequent report which noted that: “Among U.S. adults without immunocompromising conditions, VE against COVID-19 hospitalization was higher for the Moderna vaccine (93%) than the Pfizer-BioNTech vaccine (88%) and the Janssen vaccine (71%).”

Consequently, the conclusion that, “These findings reaffirm the high protection of COVID-19 vaccines against moderate and severe COVID-19 resulting in , [urgent clinic], and hospital visits” is too general, as it suggests that these three vaccines are similarly “high” in terms of the level of effectiveness against hospitalization, which is not supported by the statistical data. Notably, the 95% CI for Janssen is very wide, and hence not precise, and therefore, it may not be a reliable estimate of VE in this context. Specifically, this wider 95% CI is a function of the relatively small number of participants in the analysis who received the Janssen vaccine. It’s important to note, however, that the Moderna, Pfizer-BioNTech, and Janssen vaccines do show evidence of “continued strong protection against COVID-19-associated hospitalization and death.”

The 95% CI is a probabilistic statement of uncertainty, which is necessary for meaningful determination of vaccine efficacy and effectiveness, however, it is widely misunderstood, misinterpreted, and underreported. Also, while acknowledging the constraints to generalizability, these data add to the emerging and consistent evidence of variability in the effectiveness of the COVID-19 vaccines, amidst growing concerns about waning vaccine-induced immunity and an increase in the number of breakthrough cases during the Delta wave. Consistent in this regard, last week, an FDA advisory panel “recommended emergency use authorization (EUA) of a booster dose of the Pfizer/BioNTech COVID-19 vaccine (Comirnaty) for individuals 65 and older, and those judged to be at high risk of severe COVID-19,” and yesterday, FDA authorized a third dose of the Pfizer vaccine for these groups. As we move forward with the booster dose in certain populations, the variability in the effectiveness of the COVID-19 vaccines is sure to fuel the debate about the need for heterologous boosting, also known as mix and match.

Rossi A. Hassad, PhD, MPH, is an epidemiologist and professor at Mercy College, in Dobbs Ferry, New York. He is a Fellow of the American College of Epidemiology and a Chartered Statistician of Britain’s Royal Statistical Society.

Note: This article have been indexed to our site. We do not claim legitimacy, ownership or copyright of any of the content above. To see the article at original source Click Here

Related Posts
“Why are you ignoring me?”  50-year-old arrested for murdering neighborhood junior thumbnail

“Why are you ignoring me?” 50-year-old arrested for murdering neighborhood junior

한밤 중에 함께 술을 마시던 동네 후배를 흉기로 찔러 살해한 혐의를 받는 50대가 경찰에 붙잡혔다. 부산 북부경찰서는 29일 50대 A씨를 살인 혐의로 붙잡아 조사 중이라고 밝혔다. 경찰에 따르면 A씨는 지난 28일 오후 11시쯤 부산 북구의 한 아파트에서 동네 후배인 50대 B씨와 함께 술을 마시던 중 말다툼을 했고 자신을 무시한다는 이유로 집 안에 있던 흉기로 B씨를 찔러 살해한 혐의를…
Read More
닳아버린 신발 밑창…학대 피해 아이의 첫 번째 소원은 ‘엄마랑 살지 않기’ thumbnail

닳아버린 신발 밑창…학대 피해 아이의 첫 번째 소원은 ‘엄마랑 살지 않기’

출생과 동시에 입양된 A군, 초등학교 4학년이던 2020년 12월 경찰서 찾아 학대 피해 사실 밝혀 / A군의 양부모는 아동복지법 위반 혐의로 최근 재판 / 공혜정 대한아동학대방지협회 대표, “A군에게는 어른들이 자기를 지켜줄 거라는 믿음이 없었다” 양부모의 학대에 시달리던 A군이 공혜정 대한아동학대방지협회 대표와 만났을 때 신고 있던 신발의 밑창. 대한아동학대방지협회 제공 양부모의 학대에 시달리던 초등학교 4학년 A군이 2020년…
Read More
Index Of News