Abstract
Objective: To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using machine-learning methods in the field of oncology.
Study design and setting: We conducted a systematic review, searching the MEDLINE database between 01/12/2022 and 31/12/2022 for studies developing a multivariable prognostic model using machine-learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices.
Results: We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data-sharing statements, with 21 (46%) indicating data were available on request to the authors and 7 declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code-sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their to model to be used in practice. The use of reporting guidelines was rare: 8 studies (18%) mentioning using a reporting guideline, with 4 (10%) using the TRIPOD statement, 1 (2%) using MI-CLAIM and CONSORT-AI, 1 (2%) using STROBE, 1 (2%) using STARD, and 1 (2%) using TREND.
Conclusion: The adoption of open science principles in oncology studies developing prognostic models using machine-learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science is needed for prediction research in oncology.
Study design and setting: We conducted a systematic review, searching the MEDLINE database between 01/12/2022 and 31/12/2022 for studies developing a multivariable prognostic model using machine-learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices.
Results: We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data-sharing statements, with 21 (46%) indicating data were available on request to the authors and 7 declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code-sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their to model to be used in practice. The use of reporting guidelines was rare: 8 studies (18%) mentioning using a reporting guideline, with 4 (10%) using the TRIPOD statement, 1 (2%) using MI-CLAIM and CONSORT-AI, 1 (2%) using STROBE, 1 (2%) using STARD, and 1 (2%) using TREND.
Conclusion: The adoption of open science principles in oncology studies developing prognostic models using machine-learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science is needed for prediction research in oncology.
| Original language | English |
|---|---|
| Journal | Journal of Clinical Epidemiology |
| Early online date | 28 Oct 2023 |
| DOIs | |
| Publication status | E-pub ahead of print - 28 Oct 2023 |
Bibliographical note
Funding statement:GSC, PL, JdB, RW, and MMS are supported by Cancer Research UK (programme grant: C49297/A27294). GSC and RDR are supported by an MRC-NIHR Better Methods Better Research grant (reference: MR/V038168/1). RDR was supported by the NIHR Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham. PD was supported by Cancer Research UK (project grant: PRCPJT-Nov21\100021). The views expressed are those of the author(s) and not necessarily those of the Cancer Research UK, the NHS, the NIHR or the Department of Health and Social Care.
Keywords
- Open science
- Prognosis
- Machine learning
- Reporting
- Data sharing
- Code sharing