Automatic assessment of mathematical programming exercises with Numbas

Chris Graham


As programming has become a common feature of undergraduate mathematics degrees, there has been an increasing focus on how to teach and assess the subject to mathematicians. The potential benefits of e-assessment of basic programming exercises have many parallels with assessment in mathematics where e-assessment tools are widely used: the chance to give instant feedback to students offers an opportunity to allow students to work at their own pace, accommodating the disparate background in programming that often exists in undergraduate mathematics cohorts. And the randomisation of question content not only offers a powerful tool for practice, with students able to repeat similar problems over and over, it also can offer some protection against plagiarism in a subject where, just like a solution to some mathematical problems, student answers to identical problems are likely to be very similar. This paper considers an extension to Numbas to automatically assess programming exercises and the successful implementation of the resource in undergraduate modules using the programming languages R and Python.


Assessment, E-Assessment, Programming, Coding, Computing, Numbas

Full Text:



Ala-Mutka, K.M., 2005. A survey of automated assessment approaches for programming assignments. Computer science education, 15(2), pp.83-102.

Blank, D.S., Bourgin, D., Brown, A., Bussonnier, M., Frederic, J., Granger, B., Griffiths, T.L., Hamrick, J., Kelley, K., Pacer, M. and Page, L., 2019. nbgrader: A tool for creating and grading assignments in the Jupyter Notebook. The Journal of Open Source Education, 2(11).

Beevers, C.E. and Paterson, J.S., 2003. Automatic assessment of problem-solving skills in Mathematics. Active Learning in Higher Education, 4(2), pp.127-144.

Cloud9 and Mozilla. Ace - high performance code editor for the web [Computer software]. [Accessed 1 March 2022].

Croft, D. and England, M., 2020, January. Computing with CodeRunner at Coventry University: Automated summative assessment of Python and C++ code. In Proceedings of the 4th Conference on Computing Education Practice 2020, pp. 1-4.

Foster, B., Perfect, C. and Youd, A., 2012. A completely client-side approach to e-assessment and e-learning of mathematics and statistics. International Journal of e-Assessment, 2(2).

Graham, C., 2020. Assessment of computing in the mathematics curriculum using Numbas. MSOR Connections, 18(2).

Gwynllyw, R. and Henderson, K., 2009, August. DEWIS-a computer aided assessment system for mathematics and statistics. In CETL-MSOR Conference 2008.

Ihantola, P., Ahoniemi, T., Karavirta, V. and Seppälä, O., 2010, October. Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli calling international conference on computing education research, pp. 86-93.

Lobb, R. and Harlow, J., 2016. Coderunner: A tool for assessing computer programming skills. ACM Inroads, 7(1), pp.47-51.

Lawson-Perfect, C., Foster, W., Youd, A., Graham, C. and Stagg, G. 2021. Numbas (Version v6.0) [Computer software]. Available at: [Accessed 1 March 2022].

Perfect, C., 2015. A demonstration of Numbas, an e-assessment system for mathematical disciplines. In CAA Conference, pp. 1-8.

Pyodide contributors and Mozilla. 2019-2021. Pyodide (Version v0.19.1) [Computer software]. Available at: [Accessed 1 March 2022].

Sangwin, C., 2015. Computer aided assessment of mathematics using STACK. In Selected regular lectures from the 12th international congress on mathematical education, pp. 695-713. Springer, Cham.

Sangwin, C.J. and O’Toole, C., 2017. Computer programming in the UK undergraduate mathematics curriculum. International Journal of Mathematical Education in Science and Technology, 48(8), pp.1133-1152.

Sangwin, C.J., 2019. Automatic assessment of students’ code using CodeRunner. IMA Higher Education Teaching and Learning Series. [Accessed 1 March 2022]

Stagg, G., Graham C. and Lawson-Perfect, C., 2022, Chirun [Computer software]. Available at: [Accessed 1 March 2022].

Stagg, G. 2022. WebR [Computer software]. Available at: [Accessed 1 March 2022].

WebAssembly Community Group, 2022, WebAssembly System Interface [Computer software]. Available at: [Accessed 1 March 2022]



  • There are currently no refbacks.