External validation of the updated ADO score in COPD patients from the Birmingham COPD cohort

Background: Reviews suggest that the ADO score is the most discriminatory prognostic score for predicting mortality among chronic obstructive pulmonary disease (COPD) patients, but a full evaluation and external validation within primary care settings is critical before implementation.
Objectives: To validate the ADO score in prevalent and screen-detected primary care COPD cases at 3 years and at shorter time periods.
Patients and methods: One thousand eight hundred and ninety-two COPD cases were recruited between 2012 and 2014 from 71 United Kingdom general practices as part of the Birmingham COPD Cohort study. Cases were either on the practice COPD register or screendetected. We validated the ADO score for predicting 3-year mortality with 1-year and 2-year mortality as secondary endpoints using discrimination (area-under-the-curve (AUC)) and calibration plots.
Results: One hundred and fifty-four deaths occurred within 3 years. The ADO score was discriminatory for predicting 3-year mortality (AUC= 0.74; 95% CI: 0.69–0.79). Similar performance was found for 1- (AUC= 0.73; 0.66–0.80) and 2-year mortality (0.72; 0.67–0.76). The ADO score showed reasonable calibration for predicting 3-year mortality (calibration slope 0.95; 0.70–1.19) but over-predicted in cases with higher predicted risks ofmortality at 1 (0.79; 0.45–1.13) and 2-year (0.79; 0.57–1.01) mortality.
Discussion: The ADO score showed promising discrimination in predicting 3-year mortality in a primary care population including screen-detected cases. It may need to be recalibrated if it is used to provide risk predictions for 1- or 2-year mortality since, in these time-periods, over-prediction was evident, especially in cases with higher predicted mortality risks.
Plain Language Summary: Prediction models are tools that can be used to provide estimates of likely outcomes, such as death, over a specified time period in individual patients. This information can then be used to inform treatment decisions. For example, the intensity of treatment (or monitoring) may be increased for those with higher individual risks. These tools are usually developed using data from one group of people. However, because other groups of people may have different characteristics, the accuracy of the tool needs to be checked in these other groups. The ADO (age, dyspnoea (i.e. breathlessness), and obstructed airways) score was developed to predict death within 3 years in people with COPD. Our aim was to check whether the ADO score is accurate in predicting the risk of death in a group of people with COPD identified in general practices in the UK. We also wanted to determine whether it was accurate for predicting the risk of death at time periods shorter than 3 years. Previous studies have shown that the ADO score distinguishes well between likelihood of being dead or alive (i.e. the discrimination of a model). In our sample of people with newly diagnosed and existing COPD in primary care, we confirmed these results. However, previous studies have not properly assessed the degree of agreement between the expected and observed individual risk of death (i.e. the calibration of a model). It is essential to report calibration in prognostic models because it tells you how accurate mortality predictions are likely to be for individual with a particular disease. We found that the ADO score over-predicts individual risk of death for periods <3 years. Unless adjusted,this reduces its usefulness for clinical decision-making. In addition, this has implications for other COPD prognostic scores that have been tested and used at shorter time periods than they were developed for.