Watching the watchmen: a statistical analysis of mark consistency across taught modules

Research output: Contribution to journalArticle

Standard

Harvard

APA

Vancouver

Author

Bibtex

@article{9d754b5e3f594ebaaad27bb6b6ecfcbb,
title = "Watching the watchmen:: a statistical analysis of mark consistency across taught modules",
abstract = "Verifying that taught modules are marked and taught to a common standard is important but doing so by comparing mean module marks is inadequate when students{\textquoteright} ability is not uniform across these modules. For example, a module taken by a group of students of above average ability may justifiably result in a high mean mark, without implying that inconsistent standards have been applied. We propose a modified version of the fixed effects regression that provides direct estimates of module mark biases while conditioning for student composition and requiring no additional, potentially confidential, information on students or staff. We describe how this modified fixed effects regression can be implemented on a set of student marks and how the results can be interpreted. Increases in student numbers and tuition fees have increased the preoccupation with, and monitoring of, marks. We show how one can generate statistics that are more informative of the biases in marking, while being explicit about their limitations.",
keywords = "Assessment standard, Module grade, Quality assurance",
author = "Ercolani, {Marco G.} and Ercolani, {Joanne S.}",
year = "2014",
month = sep
doi = "10.1016/j.iree.2014.05.001",
language = "English",
volume = "17",
pages = "17–29",
journal = "International Review of Economics Education",
issn = "1477-3880",
publisher = "Elsevier",

}

RIS

TY - JOUR

T1 - Watching the watchmen:

T2 - a statistical analysis of mark consistency across taught modules

AU - Ercolani, Marco G.

AU - Ercolani, Joanne S.

PY - 2014/9

Y1 - 2014/9

N2 - Verifying that taught modules are marked and taught to a common standard is important but doing so by comparing mean module marks is inadequate when students’ ability is not uniform across these modules. For example, a module taken by a group of students of above average ability may justifiably result in a high mean mark, without implying that inconsistent standards have been applied. We propose a modified version of the fixed effects regression that provides direct estimates of module mark biases while conditioning for student composition and requiring no additional, potentially confidential, information on students or staff. We describe how this modified fixed effects regression can be implemented on a set of student marks and how the results can be interpreted. Increases in student numbers and tuition fees have increased the preoccupation with, and monitoring of, marks. We show how one can generate statistics that are more informative of the biases in marking, while being explicit about their limitations.

AB - Verifying that taught modules are marked and taught to a common standard is important but doing so by comparing mean module marks is inadequate when students’ ability is not uniform across these modules. For example, a module taken by a group of students of above average ability may justifiably result in a high mean mark, without implying that inconsistent standards have been applied. We propose a modified version of the fixed effects regression that provides direct estimates of module mark biases while conditioning for student composition and requiring no additional, potentially confidential, information on students or staff. We describe how this modified fixed effects regression can be implemented on a set of student marks and how the results can be interpreted. Increases in student numbers and tuition fees have increased the preoccupation with, and monitoring of, marks. We show how one can generate statistics that are more informative of the biases in marking, while being explicit about their limitations.

KW - Assessment standard

KW - Module grade

KW - Quality assurance

UR - https://www.sciencedirect.com/science/article/pii/S1477388015000043

U2 - 10.1016/j.iree.2014.05.001

DO - 10.1016/j.iree.2014.05.001

M3 - Article

VL - 17

SP - 17

EP - 29

JO - International Review of Economics Education

JF - International Review of Economics Education

SN - 1477-3880

ER -