Summarising and validating test accuracy results across multiple studies for use in clinical practice

Research output: Contribution to journalArticlepeer-review

Standard

Summarising and validating test accuracy results across multiple studies for use in clinical practice. / Riley, Richard D; Ahmed, Ikhlaaq; Debray, Thomas P A; Willis, Brian H; Noordzij, J Pieter; Higgins, Julian P T; Deeks, Jonathan J.

In: Statistics in Medicine, Vol. 34, No. 13, 15.06.2015, p. 2081-2103.

Research output: Contribution to journalArticlepeer-review

Harvard

APA

Vancouver

Author

Bibtex

@article{3c240c7d74c045e882651c974d248ed8,
title = "Summarising and validating test accuracy results across multiple studies for use in clinical practice",
abstract = "Following a meta-analysis of test accuracy studies, the translation of summary results into clinical practice is potentially problematic. The sensitivity, specificity and positive (PPV) and negative (NPV) predictive values of a test may differ substantially from the average meta-analysis findings, because of heterogeneity. Clinicians thus need more guidance: given the meta-analysis, is a test likely to be useful in new populations, and if so, how should test results inform the probability of existing disease (for a diagnostic test) or future adverse outcome (for a prognostic test)? We propose ways to address this. Firstly, following a meta-analysis, we suggest deriving prediction intervals and probability statements about the potential accuracy of a test in a new population. Secondly, we suggest strategies on how clinicians should derive post-test probabilities (PPV and NPV) in a new population based on existing meta-analysis results and propose a cross-validation approach for examining and comparing their calibration performance. Application is made to two clinical examples. In the first example, the joint probability that both sensitivity and specificity will be >80% in a new population is just 0.19, because of a low sensitivity. However, the summary PPV of 0.97 is high and calibrates well in new populations, with a probability of 0.78 that the true PPV will be at least 0.95. In the second example, post-test probabilities calibrate better when tailored to the prevalence in the new population, with cross-validation revealing a probability of 0.97 that the observed NPV will be within 10% of the predicted NPV.",
author = "Riley, {Richard D} and Ikhlaaq Ahmed and Debray, {Thomas P A} and Willis, {Brian H} and Noordzij, {J Pieter} and Higgins, {Julian P T} and Deeks, {Jonathan J}",
note = "{\textcopyright} 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.",
year = "2015",
month = jun,
day = "15",
doi = "10.1002/sim.6471",
language = "English",
volume = "34",
pages = "2081--2103",
journal = "Statistics in Medicine",
issn = "0277-6715",
publisher = "Wiley",
number = "13",

}

RIS

TY - JOUR

T1 - Summarising and validating test accuracy results across multiple studies for use in clinical practice

AU - Riley, Richard D

AU - Ahmed, Ikhlaaq

AU - Debray, Thomas P A

AU - Willis, Brian H

AU - Noordzij, J Pieter

AU - Higgins, Julian P T

AU - Deeks, Jonathan J

N1 - © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

PY - 2015/6/15

Y1 - 2015/6/15

N2 - Following a meta-analysis of test accuracy studies, the translation of summary results into clinical practice is potentially problematic. The sensitivity, specificity and positive (PPV) and negative (NPV) predictive values of a test may differ substantially from the average meta-analysis findings, because of heterogeneity. Clinicians thus need more guidance: given the meta-analysis, is a test likely to be useful in new populations, and if so, how should test results inform the probability of existing disease (for a diagnostic test) or future adverse outcome (for a prognostic test)? We propose ways to address this. Firstly, following a meta-analysis, we suggest deriving prediction intervals and probability statements about the potential accuracy of a test in a new population. Secondly, we suggest strategies on how clinicians should derive post-test probabilities (PPV and NPV) in a new population based on existing meta-analysis results and propose a cross-validation approach for examining and comparing their calibration performance. Application is made to two clinical examples. In the first example, the joint probability that both sensitivity and specificity will be >80% in a new population is just 0.19, because of a low sensitivity. However, the summary PPV of 0.97 is high and calibrates well in new populations, with a probability of 0.78 that the true PPV will be at least 0.95. In the second example, post-test probabilities calibrate better when tailored to the prevalence in the new population, with cross-validation revealing a probability of 0.97 that the observed NPV will be within 10% of the predicted NPV.

AB - Following a meta-analysis of test accuracy studies, the translation of summary results into clinical practice is potentially problematic. The sensitivity, specificity and positive (PPV) and negative (NPV) predictive values of a test may differ substantially from the average meta-analysis findings, because of heterogeneity. Clinicians thus need more guidance: given the meta-analysis, is a test likely to be useful in new populations, and if so, how should test results inform the probability of existing disease (for a diagnostic test) or future adverse outcome (for a prognostic test)? We propose ways to address this. Firstly, following a meta-analysis, we suggest deriving prediction intervals and probability statements about the potential accuracy of a test in a new population. Secondly, we suggest strategies on how clinicians should derive post-test probabilities (PPV and NPV) in a new population based on existing meta-analysis results and propose a cross-validation approach for examining and comparing their calibration performance. Application is made to two clinical examples. In the first example, the joint probability that both sensitivity and specificity will be >80% in a new population is just 0.19, because of a low sensitivity. However, the summary PPV of 0.97 is high and calibrates well in new populations, with a probability of 0.78 that the true PPV will be at least 0.95. In the second example, post-test probabilities calibrate better when tailored to the prevalence in the new population, with cross-validation revealing a probability of 0.97 that the observed NPV will be within 10% of the predicted NPV.

U2 - 10.1002/sim.6471

DO - 10.1002/sim.6471

M3 - Article

C2 - 25800943

VL - 34

SP - 2081

EP - 2103

JO - Statistics in Medicine

JF - Statistics in Medicine

SN - 0277-6715

IS - 13

ER -