Testing the test: Comparing SEMAC and exact word scoring on the selective deletion cloze

A number of questions surround the issue of cloze tests and their various scoring methods, with proponents on both sides making contentious claims to their validity or lack there of. Before language teachers can determine which side the agree with, they should investigate the matter themselves and make an informed decision based upon their findings. This paper studies two issues: 1) The statistical correlation between Exact Word and Semantically Acceptable (SEMAC) scoring methods on a selective deletion cloze test and 2) The reliability coefficients of the selective deletion cloze test using both scoring methods under a variety of classroom conditions. In a series of four experiments, Exact Word and SEMAC scoring were found to correlated very highly, suggesting that both scoring methods are measuring the same language quality. Depending upon the pedagogic concerns of the teacher, either scoring method can be used. Care must be taken, however, as this study also finds the reliability coefficients of SEMAC scoring to be significantly lower than Exact Word scoring.