Changes in scoring of Direct Observation of Procedural Skills (DOPS) forms and the impact on competence assessment

Keith Siau, Paul Dunckley, Roland Valori, Mark Feeney, Neil D Hawkes, John T Anderson, Ian L P Beales, Christopher Wells, Siwan Thomas-Gibson, Gavin Johnson, Joint Advisory Group on Gastrointestinal Endoscopy (JAG)

Research output: Contribution to journalArticlepeer-review

17 Citations (Scopus)


BACKGROUND: Direct Observation of Procedural Skills (DOPS) is an established competence assessment tool in endoscopy. In July 2016, the DOPS scoring format changed from a performance-based scale to a supervision-based scale. We aimed to evaluate the impact of changes to the DOPS scale format on the distribution of scores in novice trainees and on competence assessment.

METHODS: We performed a prospective, multicenter (n = 276), observational study of formative DOPS assessments in endoscopy trainees with ≤ 100 lifetime procedures. DOPS were submitted in the 6-months before July 2016 (old scale) and after (new scale) for gastroscopy (n = 2998), sigmoidoscopy (n = 1310), colonoscopy (n = 3280), and polypectomy (n = 631). Scores for old and new DOPS were aligned to a 4-point scale and compared.

RESULTS: 8219 DOPS (43 % new and 57 % old) submitted for 1300 trainees were analyzed. Compared with old DOPS, the use of the new DOPS was associated with greater utilization of the lowest score (2.4 % vs. 0.9 %; P < 0.001), broader range of scores, and a reduction in competent scores (60.8 % vs. 86.9 %; P < 0.001). The reduction in competent scores was evident on subgroup analysis across all procedure types (P < 0.001) and for each quartile of endoscopy experience. The new DOPS was superior in characterizing the endoscopy learning curve by demonstrating progression of competent scores across quartiles of procedural experience.

CONCLUSIONS: Endoscopy assessors applied a greater range of scores using the new DOPS scale based on degree of supervision in two cohorts of trainees matched for experience. Our study provides construct validity evidence in support of the new scale format.

Original languageEnglish
Pages (from-to)770-778
Number of pages9
Issue number8
Early online date3 Apr 2018
Publication statusPublished - Aug 2018


Dive into the research topics of 'Changes in scoring of Direct Observation of Procedural Skills (DOPS) forms and the impact on competence assessment'. Together they form a unique fingerprint.

Cite this