Comparative Effectiveness Research (CER) seeks to compare alternative treatments and ways to deliver healthcare to inform healthcare decisions. It can provide evidence of the harms, benefits, and effectiveness of different treatment options. While efforts to inform healthcare decisions have always been important, the establishment of the Patient-Centered Outcomes Research Institute has brought national attention to this emerging field of research. As the number of studies in CER continues to grow, it is vitally important that the types of bias that exist as a function of the study design be explained.
Validity and careful consideration of the methods used in CER and what outcomes are produced are especially important in light of the continued controversy and confusion that surround this type of research. Current articles about CER often use confusing terminology for these types of biases–leading to confusion at the least and misinformed perceptions of study validity at the worst. In a Medical Care article published in April, Dr. Sebastien Haneuse lays out definitions and examples of selection bias and confounding bias in CER, with a particular emphasis on distinguishing between the two.
In CER, confounding bias occurs when the study fails to account for factors that affect which treatment is used–compromising internal validity (whether we think the effects in the study are true). Selection bias in this case refers to what happens when the study fails to take factors that affect who is in the study population into account–affecting external validity (whether we think the effects in the study will hold in different contexts/populations). Of course, each of these sources of potential bias can be much more complicated and nuanced than laid out here.
In the article, the author lays out the difference between the types of bias:
- Confounding Bias: “Why did a patient receive one particular drug over any other?”
- Selection Bias: “Why do some patients have complete data [eg on health outcomes] and others do not?”
What Haneuse points out is that both types of bias may occur in studies and that methods developed to ameliorate one source of bias may not (and generally do not) ameliorate the other. The table below, created for this post, summarizes some of the common combinations based on study design and implementation.
Potential bias from both confounding and selection should be addressed in both randomized trials and observational studies. Each might require different methods of collecting data and different variables to reduce bias. The statistical methods used to reduce bias also differ based on the sources. While not exhaustively covered in the article, there are many methods to ameliorate bias, such as those used to model selection in economics, that could be used in CER.
Interestingly, CER seems to be most often concerned with selection within a specified patient population rather than selection into the patient population. For example, in samples drawn from insurance companies, only individuals with insurance are in the study population and even have the chance to make it into the study subsample. Hopefully, future work in CER will also acknowledge this aspect of selection. Common terminology will help move the field forward to produce better results and information for patients, providers, and policy makers.
Jess Williams