To join the 3,647 smart people who subscribe to “The Cheat Sheet,” enter your e-mail address below. New Issues every Tuesday and Thursday. It’s free:
If you enjoy “The Cheat Sheet,” please consider joining the 14 amazing people who are chipping in a few bucks a month via Patreon. Or joining the 21 outstanding citizens who are now paid subscribers. Thank you!
Penn Says Integrity Violations Trended Slightly Down in 2022-23, Though Types of Violations Shifted
The headline in the coverage of the University of Pennsylvania’s latest student conduct report is that:
Academic integrity violations for attaining an "unfair advantage over fellow students" increased seven-fold at Penn last school year — fueled in part by the unauthorized use of ChatGPT.
It is true that cases in this “unfair advantage” group jumped from just seven to 53. Though I’d argue that the bigger finding is that, according to the report, integrity violation cases dropped about 10% year over year, from 266 to 237.
Before going further, Penn deserves credit for releasing this report. It’s important and shows good leadership. It’s deeply selfish and counterproductive when schools hide these things.
Anyway, in addition to the jump in “unfair advantage” cases, exam misconduct and unauthorized collaboration were also up. The biggest drop came from the ubiquitous category of “cheating” cases, which went from 121 to 81. Judging from the data, misconduct cases at Penn have more or less returned to pre-pandemic levels. The coverage linked above has a nice chart.
I don’t make much of the overall decline and place most of the cause - were I to guess - at a combination of a return to in-person instruction and assessment, and a comfort that pandemic cheating was over. One is a factor. The other is self-soothing.
What is most interesting is the spike in “unfair advantage” incidents and the reported link to ChatGPT. I buy that. If a student is caught using generative AI without permission, I can see professors not calling it cheating but an “unfair advantage.” The ways students cheat are constantly shifting. And I think it’s pretty smart of Penn to have such a general category of misconduct.
I also note that, even though “unfair advantage” cases spiked, there were still more cases involving “cheating” and that, though the numbers are lower, exam misconduct more than doubled. I’m not sure either of those indicates a significant recession in post-pandemic misconduct.
According to the reporting, a director at the school:
wrote that this year’s “unfair advantage” cases ranged from having unauthorized access to old computer science and nursing homework, having unauthorized access to other people’s papers, using ChatGPT or Chegg, lying about needing an extension, or accessing other people’s computers for homework answers or responses.
ChatGPT and Chegg.
For companies that are totally not used for cheating, they both get mentioned in cheating cases very, very always.
And kudos again to Penn for not shying away from that. Name them. Let’s know who and what we’re talking about.
Finally, though I did not see it broken down in the reporting, the article says:
The most common disciplinary sanctions during the 2022-23 academic year for academic integrity violations were probation, reprimand, and academic support.
One of the benefits of The Cheat Sheet is the ability to occasionally assist or amplify people, companies or ideas that support academic integrity. For this, space is regularly available, for free.
Instructify.ai is a promising, integrity-focused company based in Georgia (US) and they are seeking professors or programs for an important pilot of their technologies. The below is from the company, entirely.
I thank you for reviewing and considering it:
Most education technology companies with products geared toward academic integrity focus on intrusive techniques like lockdown browsers, keylogging, and video proctoring. There is now a new organization that is capable of providing the same results without intruding on the privacy of students.
Instructify.ai develops an AI Teaching Assistant known as TALIA (Teaching and Learning Intelligent Assistant) that is capable of performing multiple tasks for professors such as answering common questions students have about course logistics and content, managing how their copyright content is spread online, creating assessments from their course materials, and improving student engagement through conversations outside the classroom. TALIA’s newest capability, however, may help instructors combat collusion without the added pressure of lockdown browsers and video proctoring.
The team at Instructify has run Random Control Trials of the new collusion detection capability of TALIA in multiple courses at a large state university in Georgia with impressive results. The chart below highlights how TALIA, deployed in three classes with a total enrollment of 1,313 students, was able to catch 46 students (3.5%) colluding with one another.
They attribute the ability to do this through the rich data provided by Canvas’s New Quiz engine. TALIA is able to perform a temporal analysis of how each pair of students progresses through the quiz and flag suspicious indicators including rare and incorrect answers. If two students progress through the quiz questions at the same rate or view the same questions at the same time, TALIA increases the pair’s suspicion score. Similarly, if a pair of students have identical rare or incorrect answers on the quiz, TALIA increases their suspicion score. Ultimately, multiple indicators are rolled up into a single suspicion score that allows faculty to identify pairs (or rings) of students that may have worked together on the quiz with a high probability. The image below shows an example graphic professors will have access to from TALIA’s analysis.
This Fall, the team at Instructify is offering free pilots of TALIA’s full capability suite including the new collusion detection capability. The offer is only available until October 15 for the first 5 universities that reach out.
If you would like to learn more about TALIA, please visit us at instructify.ai or contact the founders at chirag@instructify.ai and anthony@instructify.ai.
If you have news to share or assistance to request in advancement of academic honesty, please let me know.
Though I am not hawking this particular company or this particular request, I do think it important that innovative companies and technologies get the help they need. The cheaters are, and always will be, ahead of us. So, if you can help, please do.
I also want to circle the statistics shared above. Keeping in mind that this is just collusion, a single and rather simple form of cheating, the rates of misconduct should be illuminating.
A three-and-a-half percent detected collusion rate among three courses is rather surprising. Again, that’s not Chegg, that’s not Course Hero or ChatGPT, or boring old plagiarism - this is collusion, the sharing of exam information in real time. This technology identified 46 students who were likely engaged in exam collusion. Forty-six.
If you buy that this one new tool found 46 students cheating in just one way in just three classes, consider that Penn said it had 237 academic misconduct cases across the entire school over a full year.
Granted, the “46” is suspicion and the “237” is filed cases. But these two schools have very similar enrollment sizes and the gap between this very limited 46 and 237 overall feels colossal, leading me to think that the distance between what we probably know is happening and what we actually do about it is far, far wider than even I feared.
