New Research: AI Detectors Work. Humans, Not So Much
Plus, integrity researcher says manipulating social norms can limit misconduct. Plus, more junk disguised as opinion. Plus, a correction.
Issue 253
To join the 3,674 smart people who subscribe to “The Cheat Sheet,” enter your e-mail address below. New Issues every Tuesday and Thursday. It’s free:
If you enjoy “The Cheat Sheet,” please consider joining the 14 amazing people who are chipping in a few bucks via Patreon. Or joining the 26 outstanding citizens who are now paid subscribers. Thank you!
More Research Shows that AI Detection Technology Works
Recent research from a group of scholars at English universities in southeast Asia (Singapore, Vietnam) adds more weight to the uncontroverted data showing that AI detection systems work. The researchers are:
Mike Perkins, Jasper Roe, Darius Postma, James McGaughran, and Don Hickerson
What may be really noteworthy about this research is that the team used an upgraded, commercial version of GPT, GPT-4, to generate answers to actual exam questions in business courses. The research group even prompted GPT to create text that would avoid AI detection:
During the submission creation process, the research team engaged in so-called prompt engineering techniques to obtain responses that met assessment requirements, but also aimed to evade possible detection by academic staff and AI detectors
These tactics included:
requests to add spelling or grammatical errors for authenticity
The team then submitted those AI-generated papers for grading, along with papers from actual students. They told the professors and other graders that there may be some AI-created papers in the mix and reminded the graders of academic integrity protocols. So, aside from alerting the grading teams about the possibilities, it’s a pretty real-world test.
To detect AI, the team used Turnitin’s system. And, drumroll please — Turnitin correctly identified 91% of the AI-generated papers, despite the efforts to avoid it.
From the paper:
Although Turnitin correctly identified 91% of the papers as containing AI-generated content, faculty members formally reported only 54.5% of the papers as potential cases of academic misconduct.
Don’t know about you, but I think 91% accuracy with the deployment of evasion tactics is impressive.
I also think that this should really undercut the idea that so-called “false positives” from AI detection systems — rare as they are — lead to ruining student lives. Even when told there were AI papers, and even when reminded of the integrity process, even when papers were flagged as AI, just 55% were formally submitted for review and potential action. In fact, graders were flat-out told:
to report any submission they suspected of being AI generated through the standard academic misconduct process used by the university
Only 55% did.
Further, the study authors:
wish to highlight the thorough nature of the academic misconduct process in place in the HEI where the experiment was performed. Following reports from faculty, potential cases of misconduct are investigated centrally, and proceed through several panels and discussions with students before any penalties are applied.
Exactly. And this is how it works at every school I know about.
In Issue 250, we discussed research showing that most AI-checking systems are highly accurate at finding human-created text:
when smushed together, all 14 detectors correctly identified human-written text as human-written text with 96% accuracy. Ten of the 14 systems were a perfect nine for nine.
In other words, false-positives are quite rare. And even when a positive result is indicated, false or accurate, only 55% of submissions were directed to formal inquiry, according to this study. From there, there is a review and investigation in a generally pro-student process.
Anyway, back to the new study, where it’s fair to say the research team was pleased with the results:
Turnitin’s ability to detect 91% of the generated submissions containing AI-generated content, despite the deployment of prompt engineering techniques by the research team to evade detection, is promising. As it is likely that any detection will raise suspicion of markers assessing the paper for potential academic misconduct violations, this shows that Turnitin AI detection may be a valuable tool in supporting academic integrity.
Start the stopwatch as to when that blub makes its way to the Turnitin website.
The key takeaway is that, at least in this study, machines were way better than humans at detecting AI-generated content:
The findings of this study shed light on the significant disparity in the ability of academic staff to identify AI-generated content. In total, 22 academic papers were submitted as part of the investigation, of which a little over half, 12 (54.5%), were identified as potentially AI-generated by the academic staff
That’s not great. Again, they were told AI-generated papers were in the submissions and they still were not much better than coin-flip accurate.
Finally, with this study, the authors say that Turnitin’s systems “only” flagged, on average, about 55% of an AI paper’s content as being from AI, though 100% of it was. They say:
This is in direct contrast to the claimed results that a maximum of 15% of AI written text is missed in AI reports, indicating that prompt engineering is an effective method for evading AI detection tools.
And I guess that’s the point. These evasion activities worked, but they did not work. More than half the material was, on average, still flagged. With a 91% accuracy overall. Nonetheless, worth noting.
I also found it noteworthy that the study authors struggled to use AI to even answer the test questions, saying:
it proved challenging to align the output of AI-generated content with the required word and content limits that could feasibly be produced by an NNES student, with repeated requests needing to be made to modify the content according to the research team’s specifications. However, the output provided by ChatGPT had a tendency to revert to its typical style, resist incorporating deliberate errors, and continue to provide non-existent references–a key indicator of GenAI-created content
In other words, engineering prompts to provide suitable answers, avoid detection and stop making stuff up may be quite hard. As such, it’s worth keeping an eye on.
Tinkering With Social Norms and Expectations on Misconduct Can Limit It
In an interview in Times Higher Ed (THE), Guy Curtis, from the University of Western Australia and a known authority on academic integrity, says that messaging to students that cheating is rare can limit it.
There is no question to me that he’s right.
We’ve known for some time that students are more likely to cheat if they think their peers or classmates are cheating. There are many reasons that is the case. But, the inverse may be true, says Curtis. From THE, he says:
that students can be prevailed upon to resist academic misconduct if they are convinced that it is rare in their peer groups – regardless of the accuracy of that conviction.
“Cracking down” is one way of achieving this, Dr Curtis told Times Higher Education. “If you can stop the cheating, it stops being the norm. Students see that those who do it aren’t getting away with it. Hence they don’t do it. You look around you; no one else is doing it. Suddenly that group momentum is undermined.”
That solution is tempting. But also a tad dangerous. According to the article:
But universities must avoid being blinded by their own narrative, Dr Curtis stressed, saying administrators must maintain a distinction “between the message you want to get out to students and the actions you want to be taking as a university…when students are doing the wrong thing”.
You will get no debate from me. Telling students that their peers don’t cheat is a solution, but it should not be the solution.
The Integrity Coverage that Drives Me Nuts
Maybe no one cares. But it’s exceptionally difficult to explain some of the truths about academic integrity when so much of the coverage and opinionating is so bad, so often.
Consider as an example, this piece in an outlet called New Scientist. I don’t know this publication and I confess, I did not read the whole article because a subscription is required. In truth, I tried to subscribe so I could read it, but there was a snafu with my credit card and I gave up. Judging by the headline and first paragraph, I think I can tell what it is, which is misinformed junk.
I said it. Junk.
The headline is:
Let's use AI to rethink education, instead of panicking about cheating
Foremost, this is a false choice. You can quite easily rethink education and panic about cheating. If you panic about cheating, you are in no way closed off from using AI to rethink education or just about anything else you’d like to do.
Moreover, aside from me, no one is suggesting we panic about cheating. I am. And I have been. And it has nothing to do with whether we use AI to rethink education or not.
The sub-head of the article is:
If we build and use AI effectively, we can create an education system where students are assessed on the quality and depth of their knowledge, rather than the content of an exam
Cool. Though, I am not sure how we assess “the quality and depth of their knowledge” without some kind of assessment. Maybe I’d know how we do that if I’d read the article. But I doubt it.
Based on what’s not blurred out, the commentary from a self-described “AI activist” starts with the story from Texas A&M wherein a professor messed up and asked ChatGTP whether it generated the text in his students’ exams (see Issue 211). The incident was clear user error and has zip-all to do with academic integrity.
The article is not nearly alone in setting up the narrative that AI is good and that efforts to create or enforce standards are bad — that if you do one, you must be in the way of progress. It’s fallacy. And it annoys me. And I wonder how many more of these trope articles we’re going to be subjected to.
Too many is the answer. Even one more is too many.
Department of Corrections Department
In the last Issue, I wrote that Post University, the school that has sued Course Hero, was in Long Island, NY. It’s not. It’s in Connecticut. I have no idea how or why I made that error, but an error it is. I apologize.