The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews: a survey and meta-epidemiological study

Research output: Contribution to journalArticlepeer-review

Standard

The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews : a survey and meta-epidemiological study. / Neimann Rasmussen, Lauge; Montgomery, Paul.

In: Systematic Reviews, Vol. 7, No. 1, 129, 23.08.2018.

Research output: Contribution to journalArticlepeer-review

Harvard

APA

Vancouver

Author

Bibtex

@article{eceb0c9208db4ed2b51ec4918e99ffec,
title = "The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews: a survey and meta-epidemiological study",
abstract = "BACKGROUND: Studies published in languages other than English are often neglected when research teams conduct systematic reviews. Literature on how to deal with non-English studies when conducting reviews have focused on the importance of including such studies, while less attention has been paid to the practical challenges of locating and assessing relevant non-English studies. We investigated the factors which might predict the inclusion of non-English studies in systematic reviews in the social sciences, to better understand how, when and why these are included/excluded.METHODS: We appraised all Campbell Collaboration systematic reviews (n = 123) published to July 2016, categorising each by its language inclusiveness. We sought additional information from review authors via a questionnaire and received responses concerning 47 reviews. Data were obtained for 17 factors and we explored correlations with the number of non-English studies in the reviews via statistical regression models. Additionally, we asked authors to identify factors that support or hinder the inclusion of non-English studies.RESULTS: Of 123 reviews, 108 did not explicitly exclude, and of these, 17 included non-English language studies. One factor correlated with the number of included non-English studies across all models: the number of countries in which the members of the review team work (B-value = 0.56; SE B = 0.24; 95% CI = 0.07-1.03; p = 0.02). This indicates that reviews which included non-English studies were more likely to be produced by international review teams. Our survey showed a dominance of researchers from English-speaking countries (52.9%) and review teams consisting only of team members from these countries (65.9%). The most frequently mentioned challenge to including non-English studies was a lack of resources (funding and time) followed by a lack of language resources (e.g. professional translators).CONCLUSION: Our findings may indicate a connection between the limited inclusion of non-English studies and a lack of resources, which forces review teams to rely on their limited language skills rather than the support of professional translators. If unaddressed, review teams risk ignoring key data and introduce bias in otherwise high-quality reviews. However, the validity and interpretation of our findings should be further assessed if we are to tackle the challenges of dealing with non-English studies.",
keywords = "Campbell Collaboration, Country bias, Language bias, Location bias, Systematic reviews",
author = "{Neimann Rasmussen}, Lauge and Paul Montgomery",
year = "2018",
month = aug,
day = "23",
doi = "10.1186/s13643-018-0786-6",
language = "English",
volume = "7",
journal = "Systematic Reviews",
issn = "2046-4053",
publisher = "BioMed Central Ltd",
number = "1",

}

RIS

TY - JOUR

T1 - The prevalence of and factors associated with inclusion of non-English language studies in Campbell systematic reviews

T2 - a survey and meta-epidemiological study

AU - Neimann Rasmussen, Lauge

AU - Montgomery, Paul

PY - 2018/8/23

Y1 - 2018/8/23

N2 - BACKGROUND: Studies published in languages other than English are often neglected when research teams conduct systematic reviews. Literature on how to deal with non-English studies when conducting reviews have focused on the importance of including such studies, while less attention has been paid to the practical challenges of locating and assessing relevant non-English studies. We investigated the factors which might predict the inclusion of non-English studies in systematic reviews in the social sciences, to better understand how, when and why these are included/excluded.METHODS: We appraised all Campbell Collaboration systematic reviews (n = 123) published to July 2016, categorising each by its language inclusiveness. We sought additional information from review authors via a questionnaire and received responses concerning 47 reviews. Data were obtained for 17 factors and we explored correlations with the number of non-English studies in the reviews via statistical regression models. Additionally, we asked authors to identify factors that support or hinder the inclusion of non-English studies.RESULTS: Of 123 reviews, 108 did not explicitly exclude, and of these, 17 included non-English language studies. One factor correlated with the number of included non-English studies across all models: the number of countries in which the members of the review team work (B-value = 0.56; SE B = 0.24; 95% CI = 0.07-1.03; p = 0.02). This indicates that reviews which included non-English studies were more likely to be produced by international review teams. Our survey showed a dominance of researchers from English-speaking countries (52.9%) and review teams consisting only of team members from these countries (65.9%). The most frequently mentioned challenge to including non-English studies was a lack of resources (funding and time) followed by a lack of language resources (e.g. professional translators).CONCLUSION: Our findings may indicate a connection between the limited inclusion of non-English studies and a lack of resources, which forces review teams to rely on their limited language skills rather than the support of professional translators. If unaddressed, review teams risk ignoring key data and introduce bias in otherwise high-quality reviews. However, the validity and interpretation of our findings should be further assessed if we are to tackle the challenges of dealing with non-English studies.

AB - BACKGROUND: Studies published in languages other than English are often neglected when research teams conduct systematic reviews. Literature on how to deal with non-English studies when conducting reviews have focused on the importance of including such studies, while less attention has been paid to the practical challenges of locating and assessing relevant non-English studies. We investigated the factors which might predict the inclusion of non-English studies in systematic reviews in the social sciences, to better understand how, when and why these are included/excluded.METHODS: We appraised all Campbell Collaboration systematic reviews (n = 123) published to July 2016, categorising each by its language inclusiveness. We sought additional information from review authors via a questionnaire and received responses concerning 47 reviews. Data were obtained for 17 factors and we explored correlations with the number of non-English studies in the reviews via statistical regression models. Additionally, we asked authors to identify factors that support or hinder the inclusion of non-English studies.RESULTS: Of 123 reviews, 108 did not explicitly exclude, and of these, 17 included non-English language studies. One factor correlated with the number of included non-English studies across all models: the number of countries in which the members of the review team work (B-value = 0.56; SE B = 0.24; 95% CI = 0.07-1.03; p = 0.02). This indicates that reviews which included non-English studies were more likely to be produced by international review teams. Our survey showed a dominance of researchers from English-speaking countries (52.9%) and review teams consisting only of team members from these countries (65.9%). The most frequently mentioned challenge to including non-English studies was a lack of resources (funding and time) followed by a lack of language resources (e.g. professional translators).CONCLUSION: Our findings may indicate a connection between the limited inclusion of non-English studies and a lack of resources, which forces review teams to rely on their limited language skills rather than the support of professional translators. If unaddressed, review teams risk ignoring key data and introduce bias in otherwise high-quality reviews. However, the validity and interpretation of our findings should be further assessed if we are to tackle the challenges of dealing with non-English studies.

KW - Campbell Collaboration

KW - Country bias

KW - Language bias

KW - Location bias

KW - Systematic reviews

U2 - 10.1186/s13643-018-0786-6

DO - 10.1186/s13643-018-0786-6

M3 - Article

C2 - 30139391

VL - 7

JO - Systematic Reviews

JF - Systematic Reviews

SN - 2046-4053

IS - 1

M1 - 129

ER -