Evaluation criteria to assess the value of identification sources for horizon scanning

Joanna Smith, Alison Cook, Claire Packer

Research output: Contribution to journalArticlepeer-review

11 Citations (Scopus)
649 Downloads (Pure)


Objectives: The English National Horizon Scanning Centre routinely scans thirty-five sources to identify new and emerging health technologies. The aim of the study was to develop and apply evaluation criteria and scores to assess the value of sources, and to identify a cutoff score below which sources would be recommended for removal from routine horizon scanning. Criteria to evaluate each source scanned could result in a more efficient approach in the selection process. Methods: Evaluation criteria were developed following a review of the literature and discussions with horizon analysts. Proposed criteria were piloted on a random selection of six sources, and then applied to all thirty-five sources. The criteria were assessed using the Analytic Hierarchy Process. Results: Eight criteria were identified as being most relevant for assessing the value of scanning sources. The three most important (primary) criteria were coverage (approximate percentage of relevant information), quality (reliable, accurate, objective), and efficiency (estimated time to identify one potentially significant health technology or other relevant information). Seven sources fell beneath the cutoff score and were recommended for removal from routine scanning. Conclusions: The criteria were considered useful in the assessment of current sources, and have the potential to be used to assess new ones. These criteria may be useful for other horizon scanning centers to pilot and validate.
Original languageEnglish
Pages (from-to)348-353
Number of pages6
JournalInternational Journal of Technology Assessment in Health Care
Issue number3
Publication statusPublished - 1 Jul 2010


  • Horizon scanning systems
  • Priority setting
  • Health technology assessment
  • New and emerging health technologies


Dive into the research topics of 'Evaluation criteria to assess the value of identification sources for horizon scanning'. Together they form a unique fingerprint.

Cite this