Rating the quality of a body of evidence on the effectiveness of health and social interventions: a systematic review and mapping of evidence domains.

Research output: Contribution to journalArticle

Authors

External organisations

  • Center for International Health, Ludwig-Maximilians Universität Munchen (CIHLMU)
  • RAND Corporation, Santa Monica

Abstract

Introduction: Rating the quality of a body of evidence is an increasingly common
component of research syntheses on intervention effectiveness. This study
sought to identify and examine existing systems for rating the quality of a body
of evidence on the effectiveness of health and social interventions.

Methods: We used a multi-component search strategy to search for full‐length
reports of systems for rating the quality of a body of evidence on the effectiveness of health and social interventions published in English from 1995 onward. Two independent reviewers extracted data from each eligible system on the evidence domains included, as well as the development and dissemination processes for each system.

Results: Seventeen systems met our eligibility criteria. Across systems, we
identified 13 discrete evidence domains: study design, study execution, consistency, measures of precision, directness, publication bias, magnitude of effect, dose‐response, plausible confounding, analogy, robustness, applicability, and coherence. We found little reporting of rigorous procedures in the development and dissemination of evidence rating systems.

Conclusion: We identified 17 systems for rating the quality of a body of evidence
on intervention effectiveness across health and social policy. Existing systems
vary greatly in the domains they include and how they operationalize
domains, and most have important limitations in their development and dissemination.

The construct of the quality of the body of evidence was defined
in a few systems largely extending the Grading of Recommendations Assessment, Development, and Evaluation approach. Grading of

Recommendations
Assessment, Development, and Evaluation was found to be unique in its comprehensive guidance, rigorous development, and dissemination strategy.

Details

Original languageEnglish
Pages (from-to)224-242
Number of pages20
JournalResearch Synthesis Methods
Volume9
Issue number2
Early online date18 Jan 2018
Publication statusPublished - 9 Jun 2018