Test security in standardised patient assessments during high stakes GP training interviews

BACKGROUND: Selection interviews for GP training places are high-stakes interviews that demand fair and robust assessment methods. The aim of this study was to evaluate whether the results of standardised patient assessments on consecutive days of the week using the same scenarios were equivalent. We were concerned that candidates later in the week may receive information from their colleagues and potentially gain an unfair advantage by discovering the content of the exercise. METHODS: Anonymised data were obtained from 221 candidates who undertook the standardised patient assessment at the West Midlands Deanery, resulting in an interactive skills score, on five consecutive days in October 2005. The relationship between interactive skills scores and screening test score, country of graduation, gender and day of assessment was analysed using a linear model with identity link and normal error. The presence of statistical outliers was assessed. RESULTS: Analysis of the relationship between interactive skills score and screening test score, country of graduation, gender and day of assessment showed a significant association between overall score with gender and country of graduation only. The results indicated that females and those candidates who graduated in the UK on average achieved higher scores. There was no significant association between interactive skills score and day of assessment and there were no statistical outliers. CONCLUSIONS: On average candidate performance in the simulated patient exercise of GP training interviews was not influenced by the day of the assessment, indicating that, overall, candidates later in the week did not gain any advantage.