AERC Hosts: A/Prof Joshua McGrane 'Can GenAI make the (constructed response) grade?'
Share via
Evaluating the use and fine-tuning of LLMs for the automated scoring of secondary school Science and English exam short-answer tasks.
Abstract: How do generative AI models, including both closed and open-weights models, perform on short-answer exam scoring in secondary English and Science? Using over 6,000 student responses, prompt-based approaches are compared with fine-tuned models to evaluate gains in accuracy and reliability. Results show that both optimal prompting strategies and fine-tuning, even when using small tuning datasets, can make these models perform comparably to human raters, raising interesting questions about their future role in classroom, school and system-level assessment.
RSVP: AERC-info@unimelb.edu.au
| Time | 12-1 pm AEST |
| Date | 10 June 2025 |
| Location | Faculty of Education, Level 9 Conference Room (915), 100 Leicester Street, Carlton Victoria 3053 (or join via Zoom https://go.unimelb.edu.au/qr6p Password: 188289) |
Bio:
A/Prof Joshua McGrane is Deputy Director of the Assessment and Evaluation Centre (AERC) in the Faculty of Education. His research spans the philosophical, empirical, and statistical aspects of educational assessment and measurement. His research examines the role of technology in multiple aspects of education, from pedagogy to policy. His recent work focuses on automation and Artificial Intelligence and his articles have been published in several field-leading journals at the intersection of education studies, technology-enhanced learning and media studies. His research has been funded by large international bodies such as the Australian Research Council, the European Commission, and the UK’s ESRC.