Development and analysis of a Numbas diagnostic tool for use in a mathematics refresher program




education, learning technology, maths support, diagnostic tool, mathematics refresher program, knowledge map, item analysis, Numbas


We describe the development and analysis of an online diagnostic tool implemented in the Numbas e-learning system and used in an Australian university mathematics refresher program. Following the rapid transition to online delivery of the refresher program caused by COVID-19, the diagnostic instruments and methods used within the pre-pandemic, in-person, version of the program were lost. In 2022, we undertook to revive them in a way that would honour their original diagnostic purpose but offer a more sophisticated approach utilising the Numbas diagnostic exam type. Improvement of the tool after its initial deployment has involved the use of Rasch-based item analysis and recursive refinement of the knowledge map underlying the items.

Author Biographies

Donald Shearman, Western Sydney University

Mathematics Education Support Hub

Shatha Aziz, Western Sydney University

School of Computer, Data and Mathematical Sciences

Jim Pettigrew, University of New South Wales

School of MAthematics and Statistics


Meijer, R. R. and Nering, M. L. (1999). Computerized Adaptive Testing: Overview and Introduction. Applied Psychological Measurement, 23(3), pp.187-194.

Rylands L. and Shearman D. (2022). Diagnostic tests: Purposes and two case studies. MSOR Connections, 20(3), pp.45-54.

Wu, M., Tam, H.P. and Jen, T.H. (2016). Educational measurement for applied researchers. Theory into practice. Springer Singapore.


Additional Files