Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates

Research output: Contribution to journalArticle


  • Tengbin Xiong
  • Sheetal Parekh-Bhurke
  • Yoon K Loke
  • Asmaa Abdelhamid
  • Alex J Sutton
  • Alison J Eastwood
  • Richard Holland
  • Tanya Walsh
  • Anne-Marie Glenny

Colleges, School and Institutes


Indirect comparison methods have been increasingly used to assess the effectiveness of different interventions comparatively. This study evaluated a Trial Similarity and Evidence Consistency Assessment (TSECA) framework for assessing key assumptions underlying the validity of indirect comparisons.

Study Design and Setting
We applied the TSECA framework to 94 Cochrane Systematic Reviews that provided data to compare two interventions by both direct and indirect comparisons. Using the TSECA framework, two reviewers independently assessed and scored trial similarity and evidence consistency. A detailed case study provided further insight into the usefulness and limitations of the framework proposed.

Trial similarity and evidence consistency scores obtained using the assessment framework were not associated with statistically significant inconsistency between direct and indirect estimates. The case study illustrated that the assessment framework could be used to identify potentially important differences in participants, interventions, and outcome measures between different sets of trials in the indirect comparison.

Although the overall trial similarity and evidence consistency scores are unlikely to be sufficiently accurate for predicting inconsistency between direct and indirect estimates, the assessment framework proposed in this study can be a useful tool for identifying between-trial differences that may threaten the validity of indirect treatment comparisons.


Original languageEnglish
Pages (from-to)184-91
Number of pages8
JournalJournal of Clinical Epidemiology
Issue number2
Early online date24 Nov 2012
Publication statusPublished - Feb 2013


  • Indirect treatment comparison, Mixed treatment comparison, Multiple treatment meta-analysis, Network meta-analysis, Trial similarity, Evidence consistency