Abstract
Objectives
To describe the methods used to quantify heterogeneity and to propose alternative measures to improve reporting of heterogeneity in Cochrane diagnostic test accuracy (DTA) reviews.
Study Design and Setting
Our metaepidemiological study included all DTA reviews in the Cochrane Library up to October 6th, 2019. We summarized reviews’ characteristics focusing on heterogeneity analysis. We selected reviews with a bivariate model and ≥4 studies for reanalysis. In this group, we fitted bivariate random effects models and we quantified heterogeneity by means of logit variances of sensitivity and specificity, bivariate I2, median odds ratio (OR), and the area of the 95% prediction ellipse. We provided a narrative interpretation of these measures in different scenarios.
Results
There were 124 Cochrane DTA reviews of which 91 (73%) included meta-analysis. Only in 5 meta-analyses, variances of the logit sensitivity and specificity were reported, and in 21 meta-analyses (23%), the 95% prediction ellipse was reported without any calculation of its area. We selected 60 of these 91 reviews to explore the behavior of all measures of heterogeneity. We found that most reviews described the subjective heterogeneity as moderate or extreme (n = 31/60, 52%), whereas the area of the 95% prediction ellipse and the median OR for sensitivity and specificity showed high variability; the area ranged from 5% to 97%, the median OR of sensitivity ranged from 1.13 to 10.7, and the median OROR of specificity ranged from 1.18 to 19.68.
Conclusion
Cochrane DTA reviews show a poor reporting of between-study heterogeneity. Using median OR and the area of the 95% prediction ellipse will improve reporting and interpretation of this crucial aspect of DTA meta-analysis.
To describe the methods used to quantify heterogeneity and to propose alternative measures to improve reporting of heterogeneity in Cochrane diagnostic test accuracy (DTA) reviews.
Study Design and Setting
Our metaepidemiological study included all DTA reviews in the Cochrane Library up to October 6th, 2019. We summarized reviews’ characteristics focusing on heterogeneity analysis. We selected reviews with a bivariate model and ≥4 studies for reanalysis. In this group, we fitted bivariate random effects models and we quantified heterogeneity by means of logit variances of sensitivity and specificity, bivariate I2, median odds ratio (OR), and the area of the 95% prediction ellipse. We provided a narrative interpretation of these measures in different scenarios.
Results
There were 124 Cochrane DTA reviews of which 91 (73%) included meta-analysis. Only in 5 meta-analyses, variances of the logit sensitivity and specificity were reported, and in 21 meta-analyses (23%), the 95% prediction ellipse was reported without any calculation of its area. We selected 60 of these 91 reviews to explore the behavior of all measures of heterogeneity. We found that most reviews described the subjective heterogeneity as moderate or extreme (n = 31/60, 52%), whereas the area of the 95% prediction ellipse and the median OR for sensitivity and specificity showed high variability; the area ranged from 5% to 97%, the median OR of sensitivity ranged from 1.13 to 10.7, and the median OROR of specificity ranged from 1.18 to 19.68.
Conclusion
Cochrane DTA reviews show a poor reporting of between-study heterogeneity. Using median OR and the area of the 95% prediction ellipse will improve reporting and interpretation of this crucial aspect of DTA meta-analysis.
| Original language | English |
|---|---|
| Pages (from-to) | 101-112 |
| Journal | Journal of Clinical Epidemiology |
| Volume | 131 |
| Early online date | 21 Nov 2020 |
| DOIs | |
| Publication status | Published - Mar 2021 |
Bibliographical note
Funding Information:This work has been partially supported by research grants PI19/00481, from the Instituto de Salud Carlos III, Ministerio de Ciencia e Innovacion, Spain, FEDER “Una manera de hacer Europa” and Grant PID2019-104681RB-I00, Ministerio de Ciencia e Innovación, Spain.
Keywords
- Diagnostic test accuracy
- Heterogeneity
- Meta-analysis
- Systematic review
ASJC Scopus subject areas
- Epidemiology