Risk-of-bias tools for causal inference using real-world evidence vary in their coverage of stages of study generation and core bias domains: a rapid scoping review.
Abstract
Objective
This review aimed to identify and characterise tools designed, adapted, or validated to assess risk of bias (RoB) in causal real-world evidence (RWE), with a view to informing their potential use in evidence-informed decision-making. STUDY
Design
We conducted a rapid scoping review of records published from 2015 onwards. Eligible records included studies and documents describing tools explicitly developed for RWE or tools applicable to causal inference. Tool characteristics were extracted, and their items/domains were mapped across four stages of study generation and four core bias domains.
Results
Eleven tools oriented towards causal questions were included, most of which were designed primarily for primary studies. Qualitative or categorical judgements were more common than numerical scoring. Coverage of the stages of study generation was uneven: study design was the most frequently addressed stage, whereas results presentation was the least represented. The data quality stage was addressed by ten tools, although its coverage varied across instruments. Across core bias domains, no tool was concentrated in a single domain; most distributed items across several domains, although the relative emphasis placed on selection and confounding bias varied across instruments. A substantial number of items/domains could not be classified under the predefined core bias domains.
Conclusion
Available RoB tools differ substantially in their structure, methodological emphasis, and the way they operationalise bias across stages of study generation and core bias domains. These differences should be considered when interpreting and comparing assessments conducted with different tools.