Risk of bias in overviews of reviews: A scoping review of methodological guidance and four-item checklist

Madeleine Ballard, Paul Montgomery

Research output: Contribution to journalArticlepeer-review

51 Citations (Scopus)



To assess the conditions under which employing an overview of systematic reviews is likely to lead to a high risk of bias.

Study Design

To synthesise existing guidance concerning overview practice, a scoping review was conducted. Four electronic databases were searched with a pre-specified strategy (PROSPERO 2015:CRD42015027592) ending October 2015. Included studies needed to describe or develop overview methodology. Data were narratively synthesised to delineate areas highlighted as outstanding challenges or where methodological recommendations conflict.



Twenty-four papers met the inclusion criteria. There is emerging debate regarding overlapping systematic reviews; systematic review scope; quality of included research; updating; and synthesizing and reporting results. While three functions for overviews have been proposed—identify gaps, explore heterogeneity, summarize evidence—overviews cannot perform the first; are unlikely to achieve the second and third simultaneously; and can only perform the third under specific circumstances. Namely, when identified systematic reviews meet the following four conditions: (1) include primary trials that do not substantially overlap, (2) match overview scope, (3) are of high methodological quality, and (4) are up-to-date.


Considering the intended function of proposed overviews with the corresponding methodological conditions may improve the quality of this burgeoning publication type. Copyright © 2017 John Wiley & Sons, Ltd.

Original languageEnglish
Pages (from-to)92-108
Number of pages17
JournalResearch Synthesis Methods
Issue number1
Early online date10 Jan 2017
Publication statusPublished - Mar 2017


  • umbrella review
  • meta-review
  • overview of reviews
  • scoping review
  • synthesis methods


Dive into the research topics of 'Risk of bias in overviews of reviews: A scoping review of methodological guidance and four-item checklist'. Together they form a unique fingerprint.

Cite this