With higher course enrollments in data science courses, there is more grading to do and less time to provide meaningful feedback. Also, active learning activities often involve the use of additional formative assessment, like short quizzes, that instructors may be hesitant to introduce due to the need of then devoting time/resources to grade them.
In this blog post, we will present some of the features thanks to which Gradescope , an AI-assisted tool, allowed us to increase efficiency and fairness in grading.
We will cover:
- how Gradescope can make grading easier
- how it can increase the quality of grading
- how it can improve learners’ experience of feedback
- additional strengths and limitations of Gradescope
We hope that instructors can still take away tips on grading workflow even if they do not plan on using Gradescope.
1. Easier grading
The ways in which Gradescope has simplified our grading routine essentially hinge on three characteristics:
AI-assisted and manual question grouping;
grading by selecting rubric items that can be modified dynamically;
grading by question rather than by submission.
1.1. AI-assisted and manual question grouping
Perhaps our most loved feature of Gradescope is the possibility of forming and grading answer groups rather than grading answers individually. This feature is currently available only for fixed-length assignments (Create Assignment > Homework / Problem Set > Submission Type: Templated (Fixed Length) or Create Assignment > Exam / Quiz) but we hope that it will soon be usable with all types of assignment.
Provided that a question has been answered by all students in the same space in their submissions (e.g. at the beginning of page 3), Gradescope allows forming answer groups. These are groups of similar answers to which the grader intends to assign the same mark and feedback. To form these groups, Gradescope displays the answers of all students to the same question next to each other and allows the grader to browse and group homogeneous answers. After groups have been formed, the grader can assign specific feedback to the answer group, which is then automatically applied to all answers that were grouped together.
For multiple-choice or short, numeric-answer questions, the grader can be assisted in grouping answers by an AI tool. This is a kind of supervised auto-grading system that uses image recognition to group those answers that are estimated to be identical with high confidence while leaving ungrouped the ones on which there was higher uncertainty. The work of the graders is then greatly simplified, as they will only need to confirm that the grouping done by the AI system is correct and group the (usually few) answers that the autograder left ungrouped.
1.2 Dynamic rubrics
The second aspect that greatly facilitated our work at grading is the way that grades and feedback can be assigned in Gradescope. For each question, the grader can assign a mark by selecting possibly multiple items from a rubric that can be modified as grading proceeds. For each item in the rubric, the grader specifies how many points should be added/subtracted and what feedback should be given:
We find that this method is more convenient than having to pre-specify a rubric before seeing students’ actual answers. And, very importantly, the points and feedback assigned to a rubric item can be changed at any time by the grader, and the change will automatically apply to all individual answers/answer groups for which the modified rubric items were selected. This makes it very simple to make improvements or fix mistakes, even after grades have been distributed. For example, the instructor may decide that something for which the grader removed 1 point should actually not have been penalized. Then, instead of having to go through all submissions and fix the grading mistake in each, the grader can simply increase from “-1” to “0” the points assigned to the rubric item involved and re-publish the grades.
1.3 Grading by question
Having collected all students’ submissions of an assignment made of multiple questions, one may often tend to organize their grading “by submission” - that is, going through all questions in one student’s submission and then moving to another student’s submission. Instead of organizing grading by individual student’s submissions, Gradescope arranges together answers to the same questions across all students’ submissions:
Such arrangement predisposes the grader towards working by grading all answers to the same question and then moving on to a different question. This workflow lends itself towards a more effective way of:
- organizing a grader’s work - e.g. today I am grading questions 1 and 2, and tomorrow questions 3 and 4; and
- dividing tasks among graders - e.g. Grader A grades questions 1 and 4, Grader B grades questions 2 and 3.
While a grader and multiple graders can obviously find ways to organize their work also when grading by submission, focusing on grading all answers to the same question before moving on to a different question and/or assigning all answers to a question to the same grader can help to make grading both faster and more consistent.
2. Higher-quality grading
Besides making grading simpler, the grading system implemented in Gradescope can also have positive outcomes on the quality and fairness of grading:
as mentioned above, grading by question rather than by student can help making grading more consistent across submissions and thus fairer;
especially when dealing with a large class and limited grading resources, forming first homogeneous answer groups, and then assigning them a specific feedback all at once, can allow graders to provide more detailed feedback than otherwise would be feasible to do;
the system in Gradescope does not show the name of the student whose question is being assessed, neither when grading a question individually nor when forming answer groups; blind grading is known to reduce grading bias and thus increase the fairness and quality of the grades;
regrade requests are handled on the platform, thus lifting the graders from having to discern potential cheating attempts that can occur when submissions are handed back to students physically;
in addition to providing feedback using rubric items, as shown above, the grader has also the option to leave personalized feedback. This feature can be used for various purposes, such as writing more extensive explanations for some students, asking them to attend office hours to be given an in-person explanation and maybe to practice with similar problems, or leaving a message of praise or encouragement. The system saves all distinctly personalized feedback ever given to answers to a certain question, so when the graders move on to grade other answers to the same question, they can write a new personalized feedback or simply select one among those that were previously assigned.
3. Better feedback experience
Several features of Gradescope also have the potential to improve students’ experience of feedback.
As previously discussed, answer groups and dynamic rubrics mean that an extensive feedback needs only to be written once to then be received by all students with similar answers. Moreover, students can be encouraged and motivated by receiving personalized feedback, that the graders may have more time to write if they use a grading tool that simplifies and speeds up other parts of the grading.
Another important way in which Gradescope can ameliorate students’ relationship with grading concerns is regrade requests. If enabled, after grades have been published, students can easily ask for a regrade of any of their answers, supplying an explanation of the motivation for their request. Simplifying this process can increase students’ perception of grading fairness, and can make them more likely to ask for explanations when they have not understood the reason for their score, thus potentially improving their satisfaction with the instruction.
Finally, if the process of grading is expedited, students can receive feedback more quickly. This is also likely to improve their satisfaction, and especially it provides them with better opportunities to learn from the feedback received on formative assessments.
4. Other strengths and limitations
We conclude this post with an overview of other worth-mentioning strengths and limitations of Gradescope.
Strengths
it is easy to set up student-specific time extensions that are automatically extended to all assignments of specified students for that class;
Gradescope is integrated with popular learning management systems like Canvas and Blackboard;
Markdown and Latex are supported when writing feedback associated with rubric items as well as answer keys;
group submissions are supported and easy to handle;
it can be used for assisted grading of handwritten work as well as to create possibly auto-graded online assignments;
it can be used for grading math-type of questions as well as coding;
like for all other digital tools for grading, assigning, and distributing feedback online, adopting Gradescope has the non-negligible advantage of not having to carry around large piles of homework. Even for assignments completed in a physical class, Gradescope allows to upload a unique file of submissions that have been sequentially scanned, and then automatically splits this file into individual submissions. Moreover, provided students’ names and/or id were written in a designated area, Gradescope uses AI to assist in matching individual submissions and students.
Limitations
an important limitation is that Gradescope is not free;
introducing Gradescope implies an additional interface for the students and the teaching team to learn. This investment could however be worth it, as most types of assignments (handwritten, online, theory, coding, …) can be graded in Gradescope;
Gradescope’s team is still developing new features, and we believe that the platform has not yet achieved its full potential. On the other hand, the development team appears quite receptive to its users, and it may be possible to see the desired feature be implemented after sending a request;
there currently is no “Student view”. As a work-around, to visualize a student’s interface, the teaching team can enroll themselves in the class on Gradescope as fictitious students using some non-institutional email address;
learning how to use and implementing the auto-grading for coding requires a lot of time and coding from the instructor and it is quite different for those who teach using R and R Markdown when compared to other tools such as
learnr;the auto-grading for coding currently supports Jupyter notebooks, while R Markdown files are not supported to date.
Disclaimer: the materials used for illustration only includes activities of fictitious students’.