Paper based Multiple Choice Exams with Gradescope
Several years ago the University adopted a position that any multiple choice based assessment should take place online using online tools such as Blackboard. At that time paper based MCQ exams were used in small numbers across FSE and the platform used experienced persistent issues with scanning and interpreting marks.
Fast forward to January 2023 and the number of requests for online exams is growing to a point where we are struggling to find time within the exam timetable to schedule everything. With this in mind, it’s important that we use every tool available to us to support assessment and online (or automated) assessment offers us some clear benefits in terms of marking time.
Gradescope is a relatively new piece of software across FSE, one feature that is of interest is it’s ‘Bubblesheet’ assessments. This allows for MCQ based assessments to be completed using a paper ‘Bubblesheet’ and then scanned and uploaded into Gradescope. Gradescope’s Artificial Intelligence assists in the marking and grouping of student responses.
More details on how to use Bubble sheets can be found here.
Lu Shin Wong, a Senior lecturer in Chemistry, used Gradescope Bubble sheets in a summative exam for over 200 students. We asked Lu Shin about his experiences of using the Gradescope Bubblesheets:
How long did it take to prepare the Exam using the Gradescope bubble sheets – did you require initial training/support?
It was pretty straightforward, eLearning provided a link to some general Gradescope guidance on bubble sheets. This guidance was enough to get me started on setting up the rubric and trying out the scanning of a few test questions. Overall I found this part quite easy. Writing the exam questions took about a day (we would have needed to do that for a conventional exam anyway). The Gradescope set up was probably not more than 2-3 hours, though this time was not in a single sitting. I worked with eLearning to test some scanned answers and what would happen if I altered the standard Gradescope bubble sheet.
Gradescope uses AI to analyse and process the student marks, how accurate did you find this?
I had issues trying to understand the checking process. I worked with the eLearning team who helped work through some questions I had. Most of the issues were due to us using the tool for the first time and so it took time to understand what the A.I. was telling us. I can’t recall finding any errors and if anything the AI marked the papers, especially the questions where students had crossed out initial responses, really well.
How long did you spend on the “marking” process
Now that I know how it works, it would probably not take more than an hour for 250 students, excluding moderation and checking (which we would have to do for all exams anyway).
What guidance did you provide for your students?
I uploaded an example completed bubble sheet on Blackboard and gave them some basic instructions: “You should record your answers into a standard “Gradescope” form, using only the first 24 questions and leaving other questions blank“.
On reflection, I would add further instructions. Maybe to fill the form in pencil so they can erase any answers they want to change or instructions on how to cross out the undesired answer. Crossed-out answers reduced the level of certainty in Gradescope and flagged more questions for manual checking.
Did you receive any feedback from Students about the exam format?
Not specifically. This is a module taken exclusively by FBMH students, who have MCQs for other exams, so they seemed to know what to do and it was not unusual for them.
How happy/satisfied were with the outcome of this – in terms of effort vs results
Overall very satisfied. Yes, we had some issues with troubleshooting during the grading, but it was probably less tedious and time-consuming than having to do them manually. I also contacted Gradescope tech support directly, they replied very promptly (within 24h).
Would you use Gradescope bubble sheets again?
Absolutely yes! I might even try out Gradescope marking written exams in future, though this will mean having to format the exam question papers in a different manner.
Find out more about what Gradescope can do if you are interested in implementing Gradescope for handwritten assignments or use the bubblesheet assignments we always advise having a chat with the elearning team in the first instance.
