Projects per year
Abstract
Objective
Our aim was to improve meta-analysis methods for summarising a prediction model’s performance when individual participant data are available from multiple studies for external validation.
Study design & setting
We suggest multivariate meta-analysis for jointly synthesising calibration and discrimination performance, whilst accounting for their correlation. The approach estimates a prediction model's average performance, the heterogeneity in performance across populations, and the probability of ‘good’ performance in new populations. This allows different implementation strategies (e.g. recalibration) to be compared. Application is made to a diagnostic model for deep vein thrombosis (DVT) and a prognostic model for breast cancer mortality.
Results
In both examples multivariate meta-analysis reveals that calibration performance is excellent on average, but highly heterogeneous across populations unless the model’s intercept (baseline hazard) is recalibrated. For the cancer model, the probability of 'good' performance (defined by C-statistic≥0.7 and calibration slope between 0.9 and 1.1) in a new population was 0.67 with recalibration, but 0.22 without recalibration. For the DVT model, even with recalibration there was only a 0.02 probability of 'good' performance.
Conclusion
Multivariate meta-analysis can be used to externally validate a prediction model’s calibration and discrimination performance across multiple populations, and to evaluate different implementation strategies.
Original language | English |
---|---|
Journal | Journal of Clinical Epidemiology |
Early online date | 16 May 2015 |
DOIs | |
Publication status | Published - May 2015 |
Keywords
- Risk prediction
- prognostic model
- individual participant data (IPD)
- multivariate meta-analysis
- external validation
- calibration
- discrimination
- heterogeneity
- model comparison
Fingerprint
Dive into the research topics of 'Multivariate meta-analysis of individual participant data helped externally validate the performance and implementation of a prediction model'. Together they form a unique fingerprint.Projects
- 1 Finished
-
Multivariate Meta-Analysis of Multiple Correlated Outcomes: Development and Application of Methods, with Empirical Investigation of Clinical Impact
Riley, R. (Principal Investigator), Deeks, J. (Co-Investigator) & Kenyon, S. (Co-Investigator)
1/02/13 → 31/03/16
Project: Research Councils