Artificial intelligence in COVID-19 evidence syntheses was underutilized, but impactful: a methodological study

Juan R. Tercero-Hidalgo*, Khalid S. Khan, Aurora Bueno-Cavanillas, Rodrigo Fernández-López, Juan F. Huete, Carmen Amezcua-Prieto, Javier Zamora, Juan M. Fernández-Luna

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Downloads (Pure)

Abstract

Objectives: A rapidly developing scenario like a pandemic requires the prompt production of high-quality systematic reviews, which can be automated using artificial intelligence (AI) techniques. We evaluated the application of AI tools in COVID-19 evidence syntheses. Study Design: After prospective registration of the review protocol, we automated the download of all open-access COVID-19 systematic reviews in the COVID-19 Living Overview of Evidence database, indexed them for AI-related keywords, and located those that used AI tools. We compared their journals’ JCR Impact Factor, citations per month, screening workloads, completion times (from pre-registration to preprint or submission to a journal) and AMSTAR-2 methodology assessments (maximum score 13 points) with a set of publication date matched control reviews without AI. Results: Of the 3,999 COVID-19 reviews, 28 (0.7%, 95% CI 0.47–1.03%) made use of AI. On average, compared to controls (n = 64), AI reviews were published in journals with higher Impact Factors (median 8.9 vs. 3.5, P < 0.001), and screened more abstracts per author (302.2 vs. 140.3, P = 0.009) and per included study (189.0 vs. 365.8, P < 0.001) while inspecting less full texts per author (5.3 vs. 14.0, P = 0.005). No differences were found in citation counts (0.5 vs. 0.6, P = 0.600), inspected full texts per included study (3.8 vs. 3.4, P = 0.481), completion times (74.0 vs. 123.0, P = 0.205) or AMSTAR-2 (7.5 vs. 6.3, P = 0.119). Conclusion: AI was an underutilized tool in COVID-19 systematic reviews. Its usage, compared to reviews without AI, was associated with more efficient screening of literature and higher publication impact. There is scope for the application of AI in automating systematic reviews.

Original languageEnglish
Pages (from-to)124-134
Number of pages11
JournalJournal of Clinical Epidemiology
Volume148
Early online date2 May 2022
DOIs
Publication statusPublished - Aug 2022
Externally publishedYes

Bibliographical note

Funding Information:
Funding and conflicts of interest information: JRTH thanks the University of Granada for supporting his work through a Research Initiation Grant for Undergraduate Students. There are no conflicts of interest to declare.

Publisher Copyright:
© 2022 The Authors

Keywords

  • Artificial intelligence
  • Automation
  • Bibliometrics
  • COVID-19
  • Research design
  • Systematic review

ASJC Scopus subject areas

  • Epidemiology

Fingerprint

Dive into the research topics of 'Artificial intelligence in COVID-19 evidence syntheses was underutilized, but impactful: a methodological study'. Together they form a unique fingerprint.

Cite this