Asking these 4 questions can help us identify good news when we see it, be more critical of news reports, or delay our judgement until we have more information
It’s been a busy week or so for Covid vaccines. First we heard preliminary clinical trial results from the Pfizer vaccine, then the Russian Sputnik V. This week, we heard about Moderna vaccine. All these results were shared with the media, ahead of being peer reviewed and published in a journal.
As we expect preliminary results from more vaccine trials to be released in the coming weeks and months, it’s important to understand what’s behind these announcements, what news reports don’t tell us, and what researchers don’t yet know.
This can help us identify good news when we see it, be more critical of news reports, or delay our judgement until we have more information.
1. Does the news report tell me what type of trial it is?
At this stage, trial results making headlines are generally interim results of late-stage clinical trials, known as phase 3. This is when a vaccine is given to thousands of people and tested for how well it works and whether it’s safe.
In these trials, volunteers are randomised into two study arms, the vaccine arm (people who get the actual vaccine), and the placebo arm (people who get the placebo, usually an inert substance, such as a saline injection).
So, ideally, media reports should mention how the vaccine results compare with the placebo.
2. Does the media report mention safety?
As vaccines are mainly tested on healthy volunteers, it is extremely important to demonstrate the vaccine is safe.
So any media report should mention how many people are affected by side effects, the type of side effects (common/rare, serious/minor), and whether the data monitoring committee is investigating. Not all these details are available to the public.
3. Does it mention how well the vaccine works?
Trial outcomes are measured at one or more interim time points, and at the end of the trial. The types of results making the headlines currently come from this type of interim analysis. In other words, the committee will have assessed the results so far and will have given the trial a green light to proceed.
No phase 3 clinical trial has yet reported the full analysis from tens of thousands of study participants, but this will happen over the next few weeks.
4. What else do I need to know?
Current trials are reporting whether or not a vaccine prevents Covid symptoms, not whether it prevents the infection itself.
However, a recent media report about the Pfizer vaccine says it is likely to prevent 50% of infections, as well as 90% of symptomatic Covid-19.
If the vaccine has 90% efficacy, then 10% of vaccinated people could still get the symptomatic disease. We also don’t know how long immunity lasts or if there are any long-term side effects.
All we can do now is wait patiently for the full phase 3 trial results to come in over the next few weeks.
• Vaccine efficacy describes how well the vaccine offers protection against the target disease
• One measure is based on the “attack rate”, which is the proportion of the people in the trial diagnosed with Covid. We measure the attack rate in the vaccine arm and the placebo arm separately, then divide one by the other to give the “attack rate ratio”. We then subtract the attack rate ratio from 1 to get one measure of vaccine efficacy.
• For example, if 5% of the vaccine arm are diagnosed with COVID-19, while 40% of the placebo are diagnosed, then the attack rate ratio is (5%/40%) or 0.125 or 12.5%. That gives a vaccine efficacy of 87.5% (100% - 12.5%).
• Some vaccine trials report how well the immune system responds (immunogenicity). For example, the University of Oxford/AstraZeneca trial has reported the antibody response as well as several other measures of immunogenicity.
• Some trials only report on immunogenicity. This allows the trial to be smaller, shorter, and less expensive than vaccine efficacy trials, as they use immunogenicity as a surrogate for vaccine efficacy.
• Although efficacy is the preferred endpoint for vaccine trials, some regulating authorities accept evidence of immunogenicity to authorise a vaccine.
• Vaccine effectiveness describes how well the vaccine offers protection against the target disease in the real world
• Vaccine trials usually include healthy volunteers, but often don’t tell us how well the vaccine works in children, elderly people, or those with compromised immune systems.
• Reported vaccine efficacies of 90-95%, as we’ve heard recently, may sound impressive
• However, under real-world conditions, the vaccines are likely to offer much less protection in some population groups.