Evidence of Misclassification of Drug-Event Associations Classified as Gold Standard 'Negative Controls' by the Observational Medical Outcomes Partnership (OMOP)

Introduction Pharmacovigilance includes analysis of
large databases of information on drugs and events using
algorithms that detect disproportional frequencies of associations.
In order to test such algorithms, attempts have
been made to provide canonical reference lists of so-called
‘positive controls’ and ‘negative controls’. Reference sets
with even modest levels of misclassification may result in
under- or overstatement of the performance of algorithms.
Aim We sought to determine the extent to which ‘negative
control’ drug–event pairs in the Observational Medical Outcomes
Partnership (OMOP) database are misclassified
Methods We searched the medical literature for evidence
of associations between drugs and events listed by OMOP
as negative controls.
Results The criteria used in OMOP to classify positive
and negative controls are asymmetric; drug–event associations
published only as case series or case reports are
classified as positive controls if they are cited in Drug-
Induced Diseases by Tisdale and Miller, but as negative
controls if case series or case reports exist but are not cited
in Tisdale and Miller. Of 233 drug–event pairs classified in the 2013 version of OMOP as negative controls, 21 failed
to meet pre-specified OMOP adjudication criteria; in
another 19 cases we found case reports, case series, or
observational evidence that the drug and event are associated.
Overall, OMOP misclassified, or may have misclassified,
40 (17 %) of all ‘negative controls.’
Conclusions Results from studies of the performance of
signal-detection algorithms based on the OMOP gold standard
should be viewed with circumspection, because imperfect
gold standards may lead to under/overstatement of
absolute and relative signal detection algorithm performance.
Improvements to OMOP would include omitting misclassified
drug–event pairs, assigning more specific event labels,
and using more extensive sources of information.