Abstract
This study investigates the potential benefits and challenges of using generative artificial intelligence (GAI), specifically ChatGPT-4, in summative educational assessment. The focus is on evaluating GAI's ability to perform data analysis and generate structured academic content.
The study used ChatGPT-4 to perform data analysis to mimic an academic submission. Two AI-generated submissions and four human-generated submissions were compared. Seven academics graded the human and AI submissions to evaluate the performance of AI, and its detectability.
AI-generated submissions scored, on average, 15 marks higher than human submissions (median 67 vs. 51.5). Assessors correctly identified AI submissions 57% of the time for the first submission and 29% for the second but commented on the highly polished presentation and noted structural characteristic often seen in GAI outputs. High performance and specific language and formatting patterns were key indicators of AI involvement.
The conclusions show that GAI, has significant potential for enriching learning experiences. However, there is great concern around the integrity of assessments, particularly essays and online examinations. Here we show that even data analysis projects are at risk of academic integrity issues, with students able to use GAI to complete them. Integrating AI in education must be carefully managed to address ethical considerations and maintain academic integrity.
The study used ChatGPT-4 to perform data analysis to mimic an academic submission. Two AI-generated submissions and four human-generated submissions were compared. Seven academics graded the human and AI submissions to evaluate the performance of AI, and its detectability.
AI-generated submissions scored, on average, 15 marks higher than human submissions (median 67 vs. 51.5). Assessors correctly identified AI submissions 57% of the time for the first submission and 29% for the second but commented on the highly polished presentation and noted structural characteristic often seen in GAI outputs. High performance and specific language and formatting patterns were key indicators of AI involvement.
The conclusions show that GAI, has significant potential for enriching learning experiences. However, there is great concern around the integrity of assessments, particularly essays and online examinations. Here we show that even data analysis projects are at risk of academic integrity issues, with students able to use GAI to complete them. Integrating AI in education must be carefully managed to address ethical considerations and maintain academic integrity.
| Original language | English |
|---|---|
| Pages (from-to) | 107-116 |
| Number of pages | 9 |
| Journal | Education in Practice |
| Volume | 6 |
| Issue number | 1 |
| Publication status | Published - 25 Feb 2025 |