Teaching critical appraisal skills in health care settings

BACKGROUND: Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, results and relevance to an individual's work. Within the last decade critical appraisal has been added as a topic to many medical school and UK Royal College curricula, and several continuing professional development ventures have been funded to provide further training.

OBJECTIVES: To assess the effects of teaching critical appraisal skills to health professionals, on the process of care, patient outcomes and knowledge of health professionals.

SEARCH STRATEGY: We searched The Cochrane Library (to Issue 2 2000), MEDLINE (1966 to 1997), EMBASE (1980 to 1997), Eric (1966 to 1997), Cinahl (1982 to 1997), Lisa (1976 to 1997), Sigle (1980 to 1997), Science Citation Index (1981 to 1997), PsycLit (1974 to 1997), the world-wide-web, and reference lists of articles. We also contacted major medical education centres.

SELECTION CRITERIA: Randomised trials, controlled clinical trials, controlled before and after studies and interrupted time series analyses of educational interventions teaching critical appraisal to health professionals. The outcomes were: process of care; patient mortality, quality of life, and satisfaction; and health professional knowledge/awareness based upon objective, standardised, validated instruments.

DATA COLLECTION AND ANALYSIS: Two reviewers independently extracted data and three reviewers independently assessed study quality.

MAIN RESULTS: One USA hospital-based randomised trial was included involving 44 doctors. The outcome assessed was critical appraisal knowledge. Process of care, patient health or attitude/awareness outcomes were not assessed. Critical appraisal teaching was reported to have resulted in a 25% improvement (adjusted figure) in critical appraisal knowledge in the intervention group compared to a 6% improvement in the control group, which was statistically significant (p=0.02).

REVIEWER'S CONCLUSIONS: There is evidence that critical appraisal teaching has positive effects on participants' knowledge, but as only one study met the inclusion criteria the validity of drawing general conclusions about the effects of teaching critical appraisal is debatable. There are large gaps in the evidence as to whether teaching critical appraisal impacts on decision-making or patient outcomes. It is also unclear whether the size of benefit seen is large enough to be of practical significance, or whether this varies according to participant background or teaching method. The evidence supporting all outcomes is weakened by the generally poorly designed, executed and reported studies that we found.