Peer Evaluations
Peer evaluations can be an effective tool for encouraging collaboration and improving submitted work. Peer evaluations are especially useful in an online environment as in-person students typically trade and discuss paper assignments handed back in class, but this opportunity rarely presents itself online.
Before using peer evaluations, students’ assignments tended to focus on quantity and getting full marks, making them long and difficult to mark. Enforcing best practices can just result in template-driven work, with little actual engagement in the purpose of creating to the point, succinct assignments. Once peer evaluations start students quickly realize how annoying long, superfluous assignments are to read and mark, and quickly act, often striving to make the shortest, most compact assignment in the class. Assignments like this are a pleasure to read, and help students take pride in their work.
To start peer evaluations, students must be given access to completed assignments of other students. This can be done anonymously using various digital and non-digital means, provided students follow directions and don’t leave personal identifiers on their assignments, and professors can track the work for grading. Matching students to 2 or more evaluations ensures that all students receive some level of feedback even if some students choose not to complete the evaluations. The automatic matches can be monitored in the back end, and manual re-matches can be made where in the rare case some students systematically get missed or fail to receive good feedback. The Canvas and Blackboard Learning Management Systems (LMS) have this functionality built in, allowing for a seamless transition between submission and peer evaluation. Turnitin, a common educational plagiarism detection tool, also has a peer evaluation tool called PeerMark which has some added functionality. There are also numerous external peer evaluation tools such as: Peerceptiv, Kritik, PEAR and many more.
Once peer evaluations are deployed, evaluators can start reviewing them immediately, and send feedback to students within hours of submission. With multiple evaluators reviewing an assignment submission errors like missing pages or questions, or other technicalities can be caught, allowing the submitter to quickly rectify and resubmit if necessary. This keeps the assignment fresh in the students’ mind, and makes it easier to apply the feedback. In contrast, a typical professor marking hundreds of papers can take weeks before noticing a critical submission error and notifying the student, often well beyond any potential resubmission deadline and the working memory of the students’ assignment.
Constructive commenting
To complete a peer evaluation most systems require the reviewer to leave a constructive comment and complete a rubric for the assignment. Going through assignment together in class allows students to share and discuss their answers, get grading advice — “is this technically right professor”, learn and collaborate.
Constructive comments are the main focus for effective peer evaluations. The goal is to reflect on the assignment, and their own work, and exchange ideas and approaches to the assignment. The comments should focus on encouragement, highlighting what the student did well on their assignment, with small constructive recommendations. Advising students to ask questions in their comments, like “why did you say 2 2 is 5”, rather than stating the assignment is wrong, can provide a reflective path to feedback. Additionally, having a series of questions for each student to answer, or ask, in their comments can help guide the peer evaluators to make more constructive feedback.
The grading rubric, if included, should be presented as a guide to ensure the evaluator went through the assignment carefully, and assessed each essential piece of the work. The grading rubric should focus on whether a particular task in an assignment is complete, rather than assessing it’s quality or even correctness. A simple 4 point scale of “Perfect”, “Perfect but…”, “Incomplete” and “Missing” is often enough to evaluate the work. More advanced rubrics can incorporate typical mistakes and errors and appropriate point values. Adventurous students can get more involved using half marks and rubric comments, but is generally not required.
Depending on the level of involvement from the class, and the quality of the engagement in the peer evaluation exercise, the grades from the rubrics can be used to guide the professor’s grading of the assignments. The often very small pool of evaluations per assignment can make for large variation in the grades, so special care is needed to watch for outliers that would skew the mark significantly. At least the first assignment needs to be marked rigorously by the professor to ensure the evaluated grades match the professor’s expectations and identify students not following the rubric guidelines, and provide better feedback to peer evaluators to match the professor’s marking style.
The evaluators get a few key benefits, including being exposed to others’ ideas and formatting approaches that they might not otherwise see. They can also see how different approaches and answers are still correct while being very different. This can open their world view on the subject, and move them away from the “one true answer” towards understanding and an appreciation of the subject. Some reports can encourage free form and individualized formatting, with varying levels of success from student to student. Peer evaluations allow students to share and work collectively towards what they find most pleasing to read, which by extension makes it easier and more pleasant for the professor grading.
Professors normally spend countless hours grading and giving written feedback on assignments, often learning new approaches to the material and seeing a wide variety of correct answers to the same problem. Peer evaluations allow students to see and learn from this variation in responses. For peer evaluations to be effective, there must be a series of them to allow students to “implement” this new knowledge, and see how feedback changes as they update their approach.
Evaluation sequence
Approaching peer evaluations as a series gives students a chance to reflect, learn and grow. How the series is implemented can be divided in to two main styles: stand-alone papers, and multi-part projects. In both cases, the goal is to review and share solutions and approaches, but each has a little nuance of its own.
Stand-Alone Papers
Student submissions for lab reports or other stand-alone assignments often have little in terms of content connecting them to the next one. The discussion in class will often focus on the varying solutions, and understanding how wildly different responses can still be correct. There will also be considerable time spent parsing out the nuance of “not quite correct” answers, and giving students the tools to create constructive feedback.
The writing style and formatting are often as unique as the student, and can be improved throughout the term. For stand-alone submissions the in-class discussion can include formatting techniques, and why certain styles are visually appealing and effective. This can be as simple as discussing what content does and does not need to be included with each question, but can also extend to sharing pieces or whole works of especially well done formatting. Keen and skilled students will often gladly take centre stage to demonstrate how they implemented their unique formatting style, and these will creep in to other student’s assignments as the semester progresses.
Multi-Part Projects
If a course has a multi-part project, peer evaluations can also be extremely useful to guiding the progression of the work. They allow students to share each step of their work, and get multiple sets of eyes to review their work and catch potential problems that might arise in future submissions. For example, a course with experimental design running through a multi-step project, the students can use the peer evaluations to share their goals and design proposal, get feedback on the feasibility of their project idea or design, and explore possible challenges they might face in future steps.
The in-class discussion can focus on further collaborating on these ideas, and discussing the strengths of each project. Group projects can exponentially increase the amount of feedback, as each group member may receive only one or two peer evaluations, but this is then multiplied by the size of the group. Special care needs to be taken by the professor, to ensure proper student matching is completed as some automated systems may assign evaluators to multiple students in the same group.
Grading
Without proper guidence students are liable to write “good job” on all of their peer evaluations and call it done, so it is important that students also receive feedback on the peer evaluation comments and rubrics they complete.
Initially, grading peer evaluations can be a simple check that the student has completed their assigned peer evaluation comment and/or rubric. This check can often be completed quickly by the professor in a peer evaluation dashboard provided by the system. Then as the professor grades the assignment they can also watch for, and give feedback on, any particularly poor comments or rubrics.
As the value and engagement in peer evaluations increases, professors can move to a more advanced grading system. In most platforms, Canvas included, you can download all of the peer evaluations to analyze them in greater detail. More advanced systems may give this as a dashboard, or provide AI assist. A download, or dashboard view, allows the professor to sort by evaluator, and carefully grade each students peer evaluations as a whole. Starting with a simple word count column can quickly highlight comments lacking required detail. Additionally a rubric to evaluate the peer evaluations can send clear expectations to students on the importance of the comment length, detail and quality, and give clear feedback on their performance.
Using peer evaluations in a course can greatly increase the quality and readability of the work submitted and also promote sharing and a sense of community in the classroom. Once students know and experience good feedback they will come to expect it, and may even solicit their peers for faster or better quality feedback in class. With a few well-timed peer evaluation sessions throughout a course, students can give and receive comments, try new ideas and approaches, and learn to improve their work.
InstructureCon 2021 – Sharing is Caring: Engaging Through Peer Evaluations
Presentation by me at the 2021 InstructureCon conference
