‘Phantom’ compositional effects in English school value-added measures: the consequences of random baseline measurement error

Thomas Perry

    Research output: Contribution to journalArticlepeer-review

    9 Citations (Scopus)
    185 Downloads (Pure)

    Abstract

    A compositional effect is when pupil attainment is associated with the characteristics of their peers, over and above their own individual characteristics. Pupils at academically selective schools, for example, tend to out-perform similar-ability pupils who are educated with mixed-ability peers. Previous methodological studies however have shown that ‘phantom’ compositional effects can arise purely from measurement error. Through simulations using English National Pupil Database data, this paper demonstrates that moderate rates of measurement error in baseline scores can produce a systemic bias in value-added (‘Progress’) scores that favours higher ability pupils. This bias is large enough to explain the ostensibly superior performance of selective ‘grammar’ schools that is seen in the English data and create an appreciable bias across the school performance distribution. This paper describes how such biases can arise from ‘regression attenuation’ and discusses the more general methodological implications of the results. The paper concludes that the consequences of baseline measurement error on school value-added scores is greatly mitigated by controls for school-level prior attainment, but that the English ‘Progress’ value-added measures, as they do not control for prior attainment at school-level, are seriously biased and that the observed ‘grammar school effect’ is likely to be largely or wholly spurious.
    Original languageEnglish
    Pages (from-to)239–262
    Number of pages24
    JournalResearch Papers in Education
    Volume34
    Issue number2
    Early online date10 Jan 2018
    DOIs
    Publication statusPublished - 4 Mar 2019

    Keywords

    • Measurement error
    • attenuation bias
    • compositional effects
    • Progress 8
    • school value- added
    • education policyra

    Fingerprint

    Dive into the research topics of '‘Phantom’ compositional effects in English school value-added measures: the consequences of random baseline measurement error'. Together they form a unique fingerprint.

    Cite this