Statistical Considerations for Evaluating COVID Vaccine Protection

I refer to a recent Morbidity and Mortality Weekly Report (MMWR) to highlight a deficit in the reporting of statistical data: The failure to address the implications of the 95% confidence interval (95% CI).

The report pertains to estimates of COVID-19 vaccine effectiveness (VE) against emergency department or urgent clinic encounters, and hospitalizations among adults during SARS-CoV-2 (Delta) variant predominance. Regarding VE, the 95% CI is a measure of uncertainty associated with the level of VE obtained from the actual sample studied (the point estimate). Of course, other levels of confidence can be used for interval estimates, but 95% is typical in public health and medicine.

The 95% CI is used for generalizing or making an inference about the population, and can be pragmatically defined as a range of values around the point estimate that has a 95% probability of including the true level of VE for the population. The stricter and more theoretical definition and interpretation of confidence interval is based on “the hypothetical notion of considering the results that would be obtained if the study were repeated many times.” The size of the sample or subgroup is a key determinant of the precision of the 95% CI — smaller samples result in wider confidence intervals, which are less precise.

Best practice in statistical analysis and reporting regarding vaccine efficacy and effectiveness requires that both the 95% CI and the point estimate be fully interpreted and explained. However, it is not uncommon for researchers to merely mention the 95% CI in the methodology or results section of a report without addressing its implications for decision-making, while emphasizing the point estimate. This is a grave disservice to all.

Regarding the MMWR in question, while the point estimates of VE (mRNA) are very high, the lower limit of the 95% CI plausibly indicates that a lower level of VE can be expected if repeated studies were conducted on this population, and this is largely attributable to variation in research contexts, including personal characteristics. If this is not effectively communicated to the public, should the level of VE decrease in subsequent studies, the public may incorrectly interpret this as vaccine failure, when in fact, the lower level of VE may be accounted for by the 95% confidence intervals from previous studies.

Indeed, the MMWR supports that VE during the Delta wave “was significantly higher among Moderna vaccine recipients (95%) than among Pfizer-BioNTech (80%) or Janssen (60%) vaccine recipients.” Specifically, regarding COVID-19 hospitalization, the report notes the following VE point estimates:

  • mRNA-1273 (Moderna): 95% (95% CI 92-97)
  • BNT162b2 (Pfizer-BioNTech): 80% (95% CI 73-85)
  • Ad26.COV2.S (Janssen/Johnson & Johnson): 60% (95% CI 31-77)

The lower limit of the 95% CI indicates that the true value of the level of VE can be as low as 92% for Moderna, 73% for Pfizer-BioNTech, and 31% for Janssen, which albeit relatively low, marginally achieves the lower bound of “>30%” generally required by the FDA to consider a vaccine worthy of being widely deployed. Attention to the lower limit of the 95% CI is a conservative approach to assessing VE. While the Moderna and Pfizer-BioNTech mRNA vaccines demonstrated high levels of protection against moderate and severe COVID-19 resulting in hospitalization, the Moderna vaccine outperformed the other two vaccines, and the difference was statistically and substantively significant. This trend was reinforced in a subsequent report which noted that: “Among U.S. adults without immunocompromising conditions, VE against COVID-19 hospitalization was higher for the Moderna vaccine (93%) than the Pfizer-BioNTech vaccine (88%) and the Janssen vaccine (71%).”

Consequently, the conclusion that, “These findings reaffirm the high protection of COVID-19 vaccines against moderate and severe COVID-19 resulting in , [urgent clinic], and hospital visits” is too general, as it suggests that these three vaccines are similarly “high” in terms of the level of effectiveness against hospitalization, which is not supported by the statistical data. Notably, the 95% CI for Janssen is very wide, and hence not precise, and therefore, it may not be a reliable estimate of VE in this context. Specifically, this wider 95% CI is a function of the relatively small number of participants in the analysis who received the Janssen vaccine. It’s important to note, however, that the Moderna, Pfizer-BioNTech, and Janssen vaccines do show evidence of “continued strong protection against COVID-19-associated hospitalization and death.”

The 95% CI is a probabilistic statement of uncertainty, which is necessary for meaningful determination of vaccine efficacy and effectiveness, however, it is widely misunderstood, misinterpreted, and underreported. Also, while acknowledging the constraints to generalizability, these data add to the emerging and consistent evidence of variability in the effectiveness of the COVID-19 vaccines, amidst growing concerns about waning vaccine-induced immunity and an increase in the number of breakthrough cases during the Delta wave. Consistent in this regard, last week, an FDA advisory panel “recommended emergency use authorization (EUA) of a booster dose of the Pfizer/BioNTech COVID-19 vaccine (Comirnaty) for individuals 65 and older, and those judged to be at high risk of severe COVID-19,” and yesterday, FDA authorized a third dose of the Pfizer vaccine for these groups. As we move forward with the booster dose in certain populations, the variability in the effectiveness of the COVID-19 vaccines is sure to fuel the debate about the need for heterologous boosting, also known as mix and match.

Rossi A. Hassad, PhD, MPH, is an epidemiologist and professor at Mercy College, in Dobbs Ferry, New York. He is a Fellow of the American College of Epidemiology and a Chartered Statistician of Britain’s Royal Statistical Society.

Note: This article have been indexed to our site. We do not claim legitimacy, ownership or copyright of any of the content above. To see the article at original source Click Here

Related Posts
Need Kids for Clinical Studies? Try Facebook to Reach Parents thumbnail

Need Kids for Clinical Studies? Try Facebook to Reach Parents

Social media platforms are a useful way to reach parents of children with rheumatologic diseases for purposes of clinical research, researchers suggested. Children of parents who responded to surveys distributed through social media were generally similar, in terms of diagnoses and other disease characteristics, to those enrolled in a large, formal registry of pediatric rheumatologic…
Read More
Israel will donate 2.5 million doses of Astraznica vaccine to Africa thumbnail

Israel will donate 2.5 million doses of Astraznica vaccine to Africa

ראש הממשלה, שר החוץ ושר הבריאות הודיעו אמש (ד') כי הוחלט לתרום כ-2.5 מיליון מנות חיסונים לקורונה של חברת אסטרהזניקה למדינות באפריקה ולמדינות נוספות. התרומה היא חלק מיוזמת COVAX שמוביל ה-WHO, שנועדה להעניק גישה שוויונית לחיסונים לקורונה גם למדינות מתפתחות. מנות החיסון שהוקצו לתרומה הזאת היו אמורות להגיע בימים הקרובים לישראל, אך סוכם עם הנהלת…
Read More
Switzerland will vote on this thumbnail

Switzerland will vote on this

Das Stimmvolk hat am Sonntag über verschiedenste Fragen entschieden. Hier eine Auswahl der Vorlagen auf eidgenössischer, kantonaler und kommunaler Ebene. Vier Wochen vor der Abstimmung demonstrierten in Biel Menschen für die Ehe für alle, die an diesem Sonntag angenommen wurde. Anthony Anex / KeystoneEidgenössisch Ehe für alle Das Parlament hat die Einführung der Ehe für…
Read More
Covid-22: The expert's forecast from Switzerland is stirring the networks thumbnail

Covid-22: The expert's forecast from Switzerland is stirring the networks

התבטאות של פרופ' סאי רדי (Sai Reddy), מומחה באימונולוגיה סינתטית מהאוניברסיטה הטכנית (ETH) בציריך, שווייץ, שהתראיין בסוף השבוע שעבר ליומון ההמונים השווייצרי "בליק", לפיה צפויה בשנה הבאה גירסה חדשה ומאיימת יותר של נגיף הקורונה – אותה כינה Covid22  ו"סופר קורונה", הפכה בתוך דקות ספורות ל"ויראלית" והופצה ברשתות החברתיות בשפות רבות. מומחים במדינות שונות שהתבקשו להגיב…
Read More
Index Of News
Consider making some contribution to keep us going. We are donation based team who works to bring the best content to the readers. Every donation matters.
Donate Now

Subscription Form

Liking our Index Of News so far? Would you like to subscribe to receive news updates daily?