Since the novel coronavirus was first sequenced in January 2020, scientists across the globe raced to develop a vaccine in a fraction of the time it’s taken to develop previous vaccines. Less than one year later, the UK administered its first doses of Pfizer-BioNTech’s COVID-19 vaccine outside clinical trials. The U.S. quickly followed suit, and also initiated distribution of Moderna’s vaccine. Despite this expedited process, the Pfizer-BioNTech, Moderna and AstraZeneca-Oxford vaccines show efficacy rates of 95%, 94.1% and 70% respectively, well above the 50% threshold recommended by the World Health Organization (WHO) for use and approval. Understanding what these efficacy rates mean in practice, and how they compare with historical vaccines like Hepatitis A and Ebola, gives us a sense of how we can safely protect our population against SARS-CoV2.
Primary endpoints are established prior to the start of all randomized clinical trials to provide metrics that measure product efficacy. The efficacy of COVID-19 vaccines is measured by comparing the number of symptomatic cases of COVID-19 in the experimental group versus the placebo group. For example, Pfizer reported a total of 170 laboratory-confirmed COVID-19 cases in its phase III trial. Of these, 162 cases were in the placebo group and eight cases were in the treatment group. That means that the vaccine recipients only saw 5% of disease compared to the control group, yielding an efficacy rate of 95%.
To count as a symptomatic case, trial participants must demonstrate symptoms, report them, and then test positive for the disease. By measuring the incidence of symptomatic COVID-19, vaccine candidates evaluate the prevention of disease onset rather than infection onset. Further, the current trials don’t specifically measure whether vaccination reduces the risk of viral transmission. This is aligned with the WHO’s recommendation that primary clinical trial endpoints measure disease related outcomes, such as symptoms or survival. The primary goal of most vaccines is to reduce the burden of disease, not necessarily prevent infection. It’s possible that inoculated individuals are still capable of becoming infected and transmitting the virus, potentially passing the virus to unvaccinated populations and causing disease, but additional data is needed to confirm. While it’s unclear whether current COVID-19 vaccines effectively reduce transmission, most vaccines that protect individuals from a viral disease have been found to reduce transmission of the virus, as symptoms like coughing often contribute to person-to-person infection.
Currently, no COVID-19 vaccine has been approved by the FDA, but several have achieved emergency use authorization (EUA) and are expected to be granted approval in the coming months. Even after a product achieves approval, the FDA or other regulatory agencies may continue to grant emergency authorization to other products as the approved product may have supply or use case limitations, such as for pediatric populations.
Given that the Pfizer and Moderna vaccines use efficacy measurements recommended by the scientific community and far surpass the minimum 50% efficacy threshold, these vaccines are expected to achieve approval once participants have been monitored for the required time period (figure 1). For vaccines to gain full approval, the FDA requires that they “prevent disease or decrease its severity in at least 50% of people who are vaccinated,” measured by a laboratory-confirmed case of either COVID-19 or SARS-CoV-2 infection. In addition, trial participants must be monitored for at least six months after receiving the full regimen to ensure the FDA can assess adverse events and cases of severe COVID-19, data that’s pending for both vaccines.
While clinical trial endpoints generally are consistent across vaccines with different underlying technologies, they can vary depending on the nature of the disease itself. For many infectious diseases, the most common vaccine primary endpoint is clinical disease confirmed through polymerase chain reaction, antigen-detection assays, or seroconversion (developing antibodies). For chronic infections or infections with long incubation periods like HIV or tuberculosis, infection (with or without symptomatic or clinical disease) is more commonly used as the primary endpoint in clinical trials given the longer follow-up required to measure clinical disease morbidity or mortality.
Most clinical trials for COVID-19, a largely acute illness, measure the number of disease cases. Meanwhile, the primary endpoints for many historical vaccine trials measure antibody concentrations as a surrogate endpoint for clinical protection. This implies that at a certain threshold of antibody protection, downstream disease can be prevented (figure 1). In other words, vaccine efficacy could potentially be measured when participants meet a certain antibody concentration threshold, rather than when they develop the infection. The high infection rates of the COVID-19 pandemic may have precluded the need to use antibody titers or seroconversion as a proxy for disease. However, it’s unclear whether using antibody concentrations as a surrogate endpoint could have allowed interim trial data to be collected earlier—preventing the need to wait until trial participants develop the disease—or allowed for a larger group of participants in which to evaluate vaccine efficacy. For example, for the same sample size, a greater proportion of participants could have developed antibodies versus developed the disease, allowing for potentially greater confidence in the efficacy data. Critics warn that further research may be warranted to understand the most clinically relevant and timely endpoint for this and future epidemics.
A Nature study found that antibody levels in COVID-19 patients correlate with disease severity, suggesting that measuring antibody titers potentially can represent a clinically relevant endpoint for future studies. In other words, severe infection is related to higher antibody concentration, which in turn implies clinical protection in the future. Neutralizing antibody titers were heavily correlated with disease severity and anti-spike IgG levels, while asymptomatic patients had no or low levels of antibodies.
Given that COVID-19 vaccine trials measure disease rather than infection, some scientists have proposed designing new trials that are powered to specifically test transmissibility. Pfizer reports that it expects transmission data for its vaccine to be available in first-quarter 2021, and the Moderna trial demonstrated some reduction of asymptomatic infections, but full transmission data is pending. While the primary endpoints used for these COVID-19 vaccine trials have been consistent with recommendations, measuring infection may have allowed for clarity on transmission in the backdrop of a pandemic. This is especially important given that the disease is highly contagious but manifests in a wide range of outcomes depending on the individual.
Although long-term effects remain to be seen, today’s COVID-19 vaccine candidates exhibit relatively strong efficacy rates without demonstrating major safety concerns compared to historical benchmarks. The record-breaking discovery timelines reflect how scientists, manufacturers and government agencies have collaborated to bring effective vaccines to the public. Although emergency authorization doesn’t denote approval, multiple EUAs signify that we’re one step closer to returning to “normal.”