Randomized control trials are the commonly held gold standard clinical trial design.
The randomized controlled trial (RCT) is the gold standard of clinical research. However, researchers are increasingly asking: must they be? Webster-Clark, Jonsson Funk, and Stürmer discussed single-arm trials: administering a drug to an experimental group and using real-world data (RWD) to select an untreated control cohort in place of a placebo group, in a recent Medical Care article. Single-arm trials use routinely collected data as an attractive hybrid between an observational pharmacoepidemiology study and an RCT.
Controls in a single-arm trial are sourced from real-world data sources rather than a placebo group.
Single-arm trials can suffer from bias.
Biased results can occur if there is differential misclassification of confounders between groups, a key limitation of this design. This happens when confounders are measured by different methods for individuals in each group. Prospectively recruited experimental group patients have thorough, detailed information collected about them, whereas RWD can have missing information. The direction and extent of bias introduced when adjusting for misclassified confounders depends on their prevalence and the extent of misclassification, as well as their relationship with the exposure and outcome of interest.
This design and its limitations are intriguing. Many COVID-19 vaccine trials come to an end as others continue. Single-arm trials work particularly well where a placebo control is unethical. For a COVID-19 vaccine that has already undergone safety testing, does the benefit of even weak protection against COVID-19 outweigh the potential risks of withholding a vaccine? Is it ethical to withhold a potentially effective vaccine in the midst of a deadly pandemic? On the other hand, is it ethical to experiment with new vaccines when safe, effective options currently exist and are approved? Single-arm trials may be useful in testing new vaccines beyond those already approved. RWD is collected daily, with millions of COVID-19 cases so far. Single-arm trials could evaluate the efficacy of the hundreds of vaccine candidates more quickly than RCTs. The implications of biased results for patient care, though, question the reliability of this design.
Both risks and rewards come with an observational design.
A common dilemma in observational research is limited information on confounders. Administrative claims data, for example, is less likely to accurately capture sociodemographic characteristics, possibly leading to misclassification of such covariates. This can introduce bias into single-arm designs depending on the direction of confounders’ relationships with the drug and outcome of interest.
Currently, desperate to end the pandemic, either direction of bias could be catastrophic. Bias towards no effect could lead to discarding effective vaccines, delaying the pandemic end and access to safe, effective vaccines. Bias towards reduced risk of COVID-19 when vaccinated could lead to rapid approval of a vaccine that may not work. The direction and extent of bias from single-arm trials is often not clear until compared to similar RCTs, raising the question of whether the risk of bias outweighs the benefits of quickly and efficiently evaluating a vaccine to a new disease for which only a few vaccine trials have been done.
Safely using the extensive RWD that exists would alleviate some financial and time constraints from traditional RCTs and potentially allow drugs to be tested and approved more quickly. However, we must be careful to examine the potential for confounder misclassification and other biases introduced by this design [PDF].
Are hybrid trial designs a viable option in the future of COVID-19 vaccine research?
Pressure to approve vaccines quickly should not outshine potential dangers in recommendations based on biased trial results. While single-arm trials can be excellent hybrid designs moving forward as we see the end of the pandemic, especially as claims databases expand and gain popularity in pharmacoepidemiology research, we should proceed knowing and addressing their limitations, just one of which is presented here.
Kieran Tebben