Python for Engineers Concept Inventory (PECI): Contextualized assessment of programming skills for engineering undergraduates

A key outcome of Engineering Education is computing skills for industry 4.0. The curriculum must address an increasing reliance on model-based design, simulation, and algorithmic data processing. Python is a popular, in-demand language; exhaustive online resources and tutorials exist, challenging educators to make assessment robust to online plagiarism, and to make instruction contextually relevant to engineering. How do engineering educators develop, measure, and evaluate engineering students programming skills, accelerate their learning, and improve instruction effectiveness? A Concept Inventory (CI) is a powerful standardised assessment instrument to answer this question and its development and validation is an active research topic. We present a Python for Engineers CI (PECI) which assesses concepts, and errors observed in learners through 250 Multiple Choice Questions (MCQs). It is evaluated by embedding it into an introductory undergraduate module, whose pedagogy includes contextualised learning and auto-graded interactive computing laboratory notebooks. Using a mixed-methods approach on learning data from a student cohort (n=68), we validate PECI through student learning gains, reliability analysis and qualitative feedback. Items are shown to differentiate high and low performing students, and each item contributes to assessment reliability. The work is of value to engineering educators wishing to develop standardised assessment instruments for computing skills.