Statistical Considerations for Evaluating COVID Vaccine Protection

I refer to a recent Morbidity and Mortality Weekly Report (MMWR) to highlight a deficit in the reporting of statistical data: The failure to address the implications of the 95% confidence interval (95% CI).

The report pertains to estimates of COVID-19 vaccine effectiveness (VE) against emergency department or urgent clinic encounters, and hospitalizations among adults during SARS-CoV-2 (Delta) variant predominance. Regarding VE, the 95% CI is a measure of uncertainty associated with the level of VE obtained from the actual sample studied (the point estimate). Of course, other levels of confidence can be used for interval estimates, but 95% is typical in public health and medicine.

The 95% CI is used for generalizing or making an inference about the population, and can be pragmatically defined as a range of values around the point estimate that has a 95% probability of including the true level of VE for the population. The stricter and more theoretical definition and interpretation of confidence interval is based on “the hypothetical notion of considering the results that would be obtained if the study were repeated many times.” The size of the sample or subgroup is a key determinant of the precision of the 95% CI — smaller samples result in wider confidence intervals, which are less precise.

Best practice in statistical analysis and reporting regarding vaccine efficacy and effectiveness requires that both the 95% CI and the point estimate be fully interpreted and explained. However, it is not uncommon for researchers to merely mention the 95% CI in the methodology or results section of a report without addressing its implications for decision-making, while emphasizing the point estimate. This is a grave disservice to all.

Regarding the MMWR in question, while the point estimates of VE (mRNA) are very high, the lower limit of the 95% CI plausibly indicates that a lower level of VE can be expected if repeated studies were conducted on this population, and this is largely attributable to variation in research contexts, including personal characteristics. If this is not effectively communicated to the public, should the level of VE decrease in subsequent studies, the public may incorrectly interpret this as vaccine failure, when in fact, the lower level of VE may be accounted for by the 95% confidence intervals from previous studies.

Indeed, the MMWR supports that VE during the Delta wave “was significantly higher among Moderna vaccine recipients (95%) than among Pfizer-BioNTech (80%) or Janssen (60%) vaccine recipients.” Specifically, regarding COVID-19 hospitalization, the report notes the following VE point estimates:

  • mRNA-1273 (Moderna): 95% (95% CI 92-97)
  • BNT162b2 (Pfizer-BioNTech): 80% (95% CI 73-85)
  • Ad26.COV2.S (Janssen/Johnson & Johnson): 60% (95% CI 31-77)

The lower limit of the 95% CI indicates that the true value of the level of VE can be as low as 92% for Moderna, 73% for Pfizer-BioNTech, and 31% for Janssen, which albeit relatively low, marginally achieves the lower bound of “>30%” generally required by the FDA to consider a vaccine worthy of being widely deployed. Attention to the lower limit of the 95% CI is a conservative approach to assessing VE. While the Moderna and Pfizer-BioNTech mRNA vaccines demonstrated high levels of protection against moderate and severe COVID-19 resulting in hospitalization, the Moderna vaccine outperformed the other two vaccines, and the difference was statistically and substantively significant. This trend was reinforced in a subsequent report which noted that: “Among U.S. adults without immunocompromising conditions, VE against COVID-19 hospitalization was higher for the Moderna vaccine (93%) than the Pfizer-BioNTech vaccine (88%) and the Janssen vaccine (71%).”

Consequently, the conclusion that, “These findings reaffirm the high protection of COVID-19 vaccines against moderate and severe COVID-19 resulting in , [urgent clinic], and hospital visits” is too general, as it suggests that these three vaccines are similarly “high” in terms of the level of effectiveness against hospitalization, which is not supported by the statistical data. Notably, the 95% CI for Janssen is very wide, and hence not precise, and therefore, it may not be a reliable estimate of VE in this context. Specifically, this wider 95% CI is a function of the relatively small number of participants in the analysis who received the Janssen vaccine. It’s important to note, however, that the Moderna, Pfizer-BioNTech, and Janssen vaccines do show evidence of “continued strong protection against COVID-19-associated hospitalization and death.”

The 95% CI is a probabilistic statement of uncertainty, which is necessary for meaningful determination of vaccine efficacy and effectiveness, however, it is widely misunderstood, misinterpreted, and underreported. Also, while acknowledging the constraints to generalizability, these data add to the emerging and consistent evidence of variability in the effectiveness of the COVID-19 vaccines, amidst growing concerns about waning vaccine-induced immunity and an increase in the number of breakthrough cases during the Delta wave. Consistent in this regard, last week, an FDA advisory panel “recommended emergency use authorization (EUA) of a booster dose of the Pfizer/BioNTech COVID-19 vaccine (Comirnaty) for individuals 65 and older, and those judged to be at high risk of severe COVID-19,” and yesterday, FDA authorized a third dose of the Pfizer vaccine for these groups. As we move forward with the booster dose in certain populations, the variability in the effectiveness of the COVID-19 vaccines is sure to fuel the debate about the need for heterologous boosting, also known as mix and match.

Rossi A. Hassad, PhD, MPH, is an epidemiologist and professor at Mercy College, in Dobbs Ferry, New York. He is a Fellow of the American College of Epidemiology and a Chartered Statistician of Britain’s Royal Statistical Society.

Note: This article have been indexed to our site. We do not claim legitimacy, ownership or copyright of any of the content above. To see the article at original source Click Here

Related Posts
Bayer recognized as leader on climate change for fifth consecutive year thumbnail

Bayer recognized as leader on climate change for fifth consecutive year

14 December 2022 | 15:00 PM Europe/Amsterdam Summary Bayer receives the highest CDP score for its activities and transparency in climate change / This rating confirms the company’s high commitment to sustainability / A record-breaking 18,700 companies disclosed data on environmental impacts, risks, and opportunities through CDP’s platform / 3.4 percent of companies scored on
Read More
KURZMELDUNGEN -Olympiasieger Csollany an Covid-19 gestorben thumbnail

KURZMELDUNGEN -Olympiasieger Csollany an Covid-19 gestorben

Die wichtigsten Sportmeldungen im Überblick. Eishockey: Doublette von Nico Hischier Nico Hischier Jeff Curry / Reuters(sda) Nico Hischier kommt in der NHL immer besser in Fahrt. Beim 7:4-Sieg der New Jersey Devils bei den St. Louis Blues erzielt der Walliser Center zwei Treffer und einen Assist. Beim 4:3 in der 52. Minute verwertete Hischier eine…
Read More
Study finds that cultures can protect from foodborne pathogens in cheese thumbnail

Study finds that cultures can protect from foodborne pathogens in cheese

A recent study published in LWT, found that bacterial cultures, known as protective cultures, can fight pathogens and prevent them from causing illness by hampering their ability to infect someone at several key points. Protective bacterial cultures are commercially available and are designed to control undesirable microbes in foods, including foodborne pathogens such as Listeria…
Read More
CORE 1L Infuser Flask — Made By Fressko thumbnail

CORE 1L Infuser Flask — Made By Fressko

These stylish, chemical-free, stainless steel, insulated flasks include Fressko’s 2-in-1 filter and are the ideal brew as you go, companion. Easily brew your favourite tea, create detox waters or take hot soup to a picnic. It’s one infuser bottle that does it all. This product is lightweight, vacuum sealed to ensure the contents stay hot…
Read More
한국 쇼트트랙 혼성계주 예선 1회전 충격 탈락 thumbnail

한국 쇼트트랙 혼성계주 예선 1회전 충격 탈락

한국쇼트트랙 대표팀 박장혁이 5일 중국 베이징 캐피털 실내경기장에서 열린 2022 베이징동계올림픽 혼성계주 경기 도중 넘어지며 아쉬워하고 있다. 베이징=연합뉴스 올림픽 신설 종목인 쇼트트랙 혼성계주에서 첫 금메달을 노렸던 한국 대표팀의 예선 1회전에서 충격의 탈락을 당했다. 최민정, 이유빈의 여자대표팀 선수와 황대헌, 박장혁의 남자 선수들이 호흡을 맞춘 한국은 5일 중국 베이징 캐피털 실내경기장에서 열린 2022 베이징동계올림픽 쇼트트랙 혼성계주 준준결승…
Read More
Index Of News
Consider making some contribution to keep us going. We are donation based team who works to bring the best content to the readers. Every donation matters.
Donate Now

Subscription Form

Liking our Index Of News so far? Would you like to subscribe to receive news updates daily?