Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review

Mohammed T. Hudda; Lucinda Archer; Maarten van Smeden; Karel G.M. Moons; Gary S. Collins; Ewout W. Steyerberg; Charlotte Wahlich; Johannes B. Reitsma; Richard D. Riley; Ben Van Calster; Laure Wynants

doi:10.1016/j.jclinepi.2022.12.005

Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review

Mohammed T. Hudda^*, Lucinda Archer, Maarten van Smeden, Karel G.M. Moons, Gary S. Collins, Ewout W. Steyerberg, Charlotte Wahlich, Johannes B. Reitsma, Richard D. Riley, Ben Van Calster, Laure Wynants

^*Corresponding author for this work

Applied Health Research

Research output: Contribution to journal › Article › peer-review

29 Downloads (Pure)

Abstract

Objectives: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.

Study Design and Setting: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.

Results: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.

Conclusions: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

Original language	English
Pages (from-to)	75-84
Number of pages	10
Journal	Journal of Clinical Epidemiology
Volume	154
Early online date	14 Dec 2022
DOIs	https://doi.org/10.1016/j.jclinepi.2022.12.005
Publication status	Published - Feb 2023

Keywords

Peer review
Reporting guidelines
Prediction modeling
COVID-19
TRIPOD
Adherence
Prognosis and diagnosis

Access to Document

10.1016/j.jclinepi.2022.12.005Licence: Creative Commons: Attribution (CC BY)

HuddaM2022MinimalFinal published version, 783 KBLicence: Creative Commons: Attribution (CC BY)

Cite this

Hudda, M. T., Archer, L., Smeden, M. V., Moons, K. G. M., Collins, G. S., Steyerberg, E. W., Wahlich, C., Reitsma, J. B., Riley, R. D., Calster, B. V., & Wynants, L. (2023). Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review. Journal of Clinical Epidemiology, 154, 75-84. https://doi.org/10.1016/j.jclinepi.2022.12.005

@article{2dfb1b49c3a4413ba0a41e46b1f5361c,

title = "Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review",

abstract = "Objectives: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.Study Design and Setting: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.Results: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.Conclusions: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.",

keywords = "Peer review, Reporting guidelines, Prediction modeling, COVID-19, TRIPOD, Adherence, Prognosis and diagnosis",

author = "Hudda, {Mohammed T.} and Lucinda Archer and Smeden, {Maarten van} and Moons, {Karel G.M.} and Collins, {Gary S.} and Steyerberg, {Ewout W.} and Charlotte Wahlich and Reitsma, {Johannes B.} and Riley, {Richard D.} and Calster, {Ben Van} and Laure Wynants",

year = "2023",

month = feb,

doi = "10.1016/j.jclinepi.2022.12.005",

language = "English",

volume = "154",

pages = "75--84",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier",

}

TY - JOUR

T1 - Minimal reporting improvement after peer review in reports of COVID-19 prediction models

T2 - systematic review

AU - Hudda, Mohammed T.

AU - Archer, Lucinda

AU - Smeden, Maarten van

AU - Moons, Karel G.M.

AU - Collins, Gary S.

AU - Steyerberg, Ewout W.

AU - Wahlich, Charlotte

AU - Reitsma, Johannes B.

AU - Riley, Richard D.

AU - Calster, Ben Van

AU - Wynants, Laure

PY - 2023/2

Y1 - 2023/2

N2 - Objectives: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.Study Design and Setting: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.Results: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.Conclusions: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

AB - Objectives: To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.Study Design and Setting: Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.Results: Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.Conclusions: The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

KW - Peer review

KW - Reporting guidelines

KW - Prediction modeling

KW - COVID-19

KW - TRIPOD

KW - Adherence

KW - Prognosis and diagnosis

U2 - 10.1016/j.jclinepi.2022.12.005

DO - 10.1016/j.jclinepi.2022.12.005

M3 - Article

SN - 0895-4356

VL - 154

SP - 75

EP - 84

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -

Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review

Abstract

Keywords

Access to Document

Fingerprint

Cite this