A Guide to Evidence Synthesis: 9. Risk of Bias Assessment

Risk of Bias Assessment

Risk of bias assessment (sometimes called "quality assessment" or "critical appraisal") helps to establish transparency of evidence synthesis results and findings. A risk of bias assessment is often performed for each included study in your review.  Evidence syntheses strive to eliminate bias in their findings.  Individual studies that are included in a synthesis may include biases in their results or conclusions (for example, design flaws that raise questions about validity of findings or an overestimate of intervention effect). Risk of bias assessment is generally not required for evidence synthesis methods outside of systematic reviews. However, guidelines and recommendations for evidence synthesis methods are continually evolving. Be sure to check on the most recent guidance for the method you are utilizing.

 Risk of Bias Assessment in a Discipline Outside of Human Medicine 

This Cochrane Training presentation helps to navigate the steps of Risk of Bias Assessment, but discusses health studies-based sources of bias that may not apply to other disciplines. For disciplines outside of human medicine, the Critical Appraisal Skills Programme provides checklists that can be applied to a diverse array of study types, and the Mixed Methods Appraisal Tool can be a helpful resource if the included studies use mixed methods. 

Scoping reviews don't typically include a risk of bias assessment

“A key difference between scoping reviews and systematic reviews is that the former are generally conducted to provide an overview of the existing evidence regardless of methodological quality or risk of bias (4, 5). Therefore, the included sources of evidence are typically not critically appraised for scoping reviews.”

From Tricco et al.,  PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. See the full article for more methodology guidelines specific to scoping reviews.  

How do you represent your risk of bias assessment in your review manuscript? 

Risk of bias assessments can be represented in table format in an evidence synthesis, showing each included study and how strong it is across several quality criteria for that particular study type.

If a high proportion of studies are assessed with a high risk of bias, caution should be used when interpreting results for your evidence synthesis.  More information can be found in Chapter 8 of the Cochrane Handbook.    

From Liu et al. (2015): Summary of risk of bias for RCTs (RevMan 5.3).