PISA-VET assessment: why, what and how (to continue)?

Stephan is a Full Professor of Vocational Education and Training (VET) at the Technische Universität Dresden, Germany. His research focuses on various areas of VET, such as the education of VET teachers, the evaluation of VET systems, and the assessment and development of VET students' knowledge and skills (see some references below). His aim is to conduct rigorous, evidence-based VET research to develop empirically grounded VET theories that contribute to the development of students and teachers and organisations (schools, training centres, companies) and help practitioners (policy makers, teachers, trainers) to create more effective and satisfying VET systems. In 2023, he had the pleasing task of leading an international expert group (including Australian expert Grant Petch from the Kangan Institute) on behalf of the OECD to develop a framework for a specific occupational area to be assessed in PISA-VET.

The seminar consisted of four parts: Firstly, Stephan briefly introduced the overall aim, rationale, challenges and timeline of PISA-VET, including the fact that Australia and Germany are designated data collection countries in the development phase. Secondly, he then spoke about the key elements of the automotive learner framework and how we arrived at this framework. Thirdly, he presented computerised approaches to measuring automotive learners' knowledge and skills, including video-based and simulation-based tests. In this part, I will also present empirical results on the psychometric quality of these approaches and on the knowledge and skills of German apprentices. Finally, he  discussed next steps in PISA-VET assessment and research questions that could be addressed in international collaboration, for example, between Australian and German partners.

PISA-VET assessment

Rausch, A., Greiff, S., Winther, E., Deutscher, V., Abele, S., Shackelton, J., Messenger, J., Kis, V., Madruga, R., Ferreira, T., Traminte, L., Ward, M. (in preparation). Designing an International Large-Scale Assessment of Professional Competencies and Employability Skills: Emerging Avenues and Challenges of OECD’s PISA-VET. Vocations & Learning.

Assessment and development of VET students’ knowledge and skills

Abele, S., & von Davier, M. (2019). CDMs in vocational education: Assessment and usage of diagnostic problem-solving strategies in car mechatronics. In M. von Davier & Y. S. Lee (Eds.), Handbook of diagnostic classification models (pp. 461–488). Springer International Publishing. https://doi-org.wwwdb.dbod.de/10.1007/978-3-030-05584-4_22

Abele, S. (2018). Diagnostic Problem-Solving Process in Professional Contexts: Theory and Empirical Investigation in the Context of Car Mechatronics Using Computer-Generated Log-Files. Vocations and Learning 11, 133–159. https://doi-org.wwwdb.dbod.de/10.1007/s12186-017-9183-x

Abele, S. (2016). Can diagnostic problem-solving competences of car mechatronics be validly assessed using a paper-pencil test? Journal of Technical Education (JOTED), 4, 190-211.

Meier, J., Hesse, P., Abele, S., Renkl, A., & Glogger-Frey, I. (2023). Better self-explaining backwards or forwards? Prompting self-explanation in video-based modelling examples for learning a diagnostic strategy. Instructional Science, 1-26.

Meier, J., Spliethoff, L., Hesse, P., Abele, S., Renkl, A., & Glogger-Frey, I. (2022). Promoting car mechatronics apprentices' diagnostic strategy with modeling examples: Development and evaluation of a simulation-based learning environment. Studies in Educational Evaluation, 72.

Rexhäuser, D., Radkowitsch, A., Richters, C., Glogger-Frey, I., Meier, J.  & Abele, S. (in preparation). Fostering collaborative diagnostic problem solving with worked examples. Frontiers in Psychology.

Radkowitsch, A., Rexhäuser, D., Richters, C., Glogger-Frey, I., & Abele, S. (under revision). Measuring collaborative diagnostic problem solving in automotive malfunction diagnosis: Deriving starting points for instructional support. International Journal of Computer-Supported Collaborative Learning.

Schauer, J., & Abele, S. & Etzel, J. (in preparation).  Differential dynamics in apprentices’ development of vocational competencies – (How) Do certificates, initial cognitive resources, social and migration background matter? Empirical Research in Vocational Education and Training.

Schauer, J., & Abele, S. (2023). Heterogenität in der Ausbildungsreife nicht-studienberechtigter Schulabgänger: innen – eine latente Profilanalyse auf Basis des Nationalen Bildungspanels [Lower qualified school leavers at the transition to vocational education and training - a latent profile analysis of cognitive and non-cognitive skills based on the German National Educational Panel Study]. Zeitschrift für Erziehungswissenschaft, 1-29.

Ștefӑnicӑ, F., Abele, S., Walker, F. & Nickolaus, R. (2017). Modeling, measurement and development of professional competencies in industrial-technical professions. In M. Mulder & J. Winterton (Eds.). Competence-based Vocational and Professional Education: Bridging the Worlds of Work and Education (S. 843–861). Cham: Springer International Publishing.

VET teacher education

Abele, S., Vogel-Blaschka, D., Beck, M. & Fürstenau, B. (in preparation). Measuring the general pedagogical knowledge of vocational teacher students. Journal of Vocational Education & Training.

Beck, M., Leon, A., Abele, S., & Fürstenau, B. (2022). Messung des pädagogischen Unterrichtswissens bei Studierenden der Berufs- und Wirtschaftspädagogik – Evaluation eines Instruments aus dem allgemeinbildenden Bereich [Measuring the pedagogical teaching knowledge of students of vocational and business education - evaluation of an instrument from the field of general education]. In K. Kögler, U. Weyland, & H. -H. Kremer (Hrsg.), Jahrbuch der berufs- und wirtschaftspädagogischen Forschung 2022 (S. 115-132). Verlag Barbara Budrich.