(366) The "Innovative" Way to Stop AI Cheating, Which is Comical

Plus, The Sydney Morning Herald Covers Remote Exam Proctoring Like It's Five Years Ago, Because It Was. Plus, a Class Note.

Issue 366

Subscribe below to join 4,699 (+24) other smart people who get “The Cheat Sheet.” New Issues every Tuesday and Thursday.

The Cheat Sheet is free. Although, patronage through paid subscriptions is what makes this newsletter possible. Individual subscriptions start at $8 a month ($80 annual), and institutional or corporate subscriptions are $250 a year. You can also support The Cheat Sheet by giving through Patreon.

---

The “Innovative Way” to Stop AI Cheating

Sent in by a friend of The Cheat Sheet, this story in District Administration magazine is downright funny.

This headline alone ought to tell you that you’re in for a ride:

Here’s one innovative idea to stop cheating with AI

I’m laughing so hard I can’t quite type.

The story starts:

If you’re worried about students cheating with AI at home, try encouraging them—and their teachers—to use it exclusively in the classroom. The results may surprise you.

No, really. That’s the innovative way to stop cheating — encourage them to use AI only in the classroom.

And, in fact — no, District Administration, the results will absolutely not surprise me.

The district that the story cites for this groundbreaking approach is the Jordan School District in Utah. And the story says that the Superintendent:

says there’s a solution to the biggest concern around AI—cheating.

He suggests allowing AI use exclusively during school hours.

Run that by me again?

How does that work, exactly? Jordan School District, I guess, disallows AI use during non-school hours? And they can enforce this how? I mean they must have a strong policy that says AI at school is cool, you cannot roam with AI at home. Right?

I’d question whether such a policy would work, to be honest. Because it’s obviously unenforceable on its face. There is no way any district can enforce that, or would even want to try.

But wait! Wait.

The article says the Jordan Superintendent

and his team are currently working on an AI policy and guidance.

They. Do. Not. Even. Have. A. Policy.

I have to hand it to them though — that is indeed an innovative approach. Use it at school only, but don’t have a policy or any way to plausibly enforce it even if you did. That’s genius-level stuff right there.

And I cannot say which is worse, a school district thinking and saying — out loud — that using AI in school will somehow reduce cheating. Or District Administration declaring such a thing “an innovative idea” to stop cheating.

Neither one is serious. And that’s just depressing.

Sydney Morning Herald Is All Over the Precisely Wrong Problem

The Sydney Morning Herald (SMH) has done some exceptional reporting on academic integrity and credential fraud (see Issue 302, Issue 289, or Issue 354).

In Issue 302, for example, the paper reported on a 1,000% — yes, one thousand percent — increase in serious cheating cases at one Australian university and substantial increases at others. Something to keep in mind as SMH decided to recently tell its readers about the terrible perils of exam proctoring, like it’s 2021 all over again.

In absolute seriousness, this coverage is a joke. In this case, the texting abbreviation SMH, shaking my head, is entirely fitting. And frankly, I am exhausted at having to revisit this tired story about how remote proctoring and the securing of online exams is stressful. I thought we were past this.

But here we are. Like Groundhog Day.

Here are the story’s first two paragraphs:

University students say excessive AI monitoring of their exams is creating “constant fear” and increasing stress levels during important assessments while others wait months for outcomes of cheating investigations.

AI proctors, or virtual invigilators – which were commonplace as a way of cracking down on cheating when assessments were conducted remotely during the pandemic – monitor students’ behaviour during exams by recording their screen, observing their keystrokes, using the computer microphone to listen for suspicious noises and tracking eye movement to ensure students don’t look off-screen at notes.

I can’t speak to, and never defend, delays in integrity cases. Though I am highly sympathetic to the increasing complexity and litigiousness of these cases, as well as to their skyrocketing frequency. No school in the world imagined needing a system to handle such things.

As to the rest of it — I cannot comprehend what we’re even discussing. Fear and stress, I guess. I do get that stress and fear are bad. I do not envy anyone having to deal with either, let alone both.

But let me start with, and center on, the obvious, as it relates to this entire conversation about remote exam proctoring — the only alternative is to not do it and essentially allow open-season cheating. There are small places to wiggle in the middle, but this is essentially a binary choice. You can proctor online exams, or you can have invalid exams, grades, and degrees.

In fact, there’s a growing consensus among people who think about such things that online assessments are simply unsecurable, even with proctoring. But if you do have an online assessment, proctoring is the absolute bare minimum of expected care. It’s absurd to the point of farce to not secure an online assessment. It just is. And it’s an obvious choice that the SMH ignores because, you know, stress.

And again, stress is bad. But it’s also part of adulting. Going through endless and intrusive security at any airport in the world is stressful. But I get that it’s necessary because there are bad people out there who will do bad things if given any opportunity. Yet somehow, we can get an entire article in a major daily newspaper about how the stress of security provisions is the real problem. The phrase absolutely divorced from reality comes to mind.

As for this:

recording their screen, observing their keystrokes, using the computer microphone to listen for suspicious noises and tracking eye movement to ensure students don’t look off-screen at notes.

Let me say again — we can choose to not do those things. But then people would very rightly say that any exam under such conditions is precisely worthless, that all you’d be measuring is how well students cheated.

In the story, Professor Cath Ellis, an academic integrity leader at Western Sydney University, whom I like and respect, described online proctoring tools as “creepy.” And fine. It’s creepy to have a stranger open and dig through my personal carry-on at the airport. But I ask again — who would prefer the alternative?

More from the SMH:

The high degree of surveillance is affecting student performance, says University of Technology Sydney student society technology officer Sina Afsharmehr.

“It’s definitely something that a lot of students are worried about,” he said. UTS still conducts exams virtually and uses a platform called ProctorU to closely monitor students.

“Every student that’s sitting an exam knows that there’s a risk if they look away from their screen [because it is flagged as suspicious behaviour], and I think for a lot of students that does cause a lot of stress.”

First, please note “high degree of surveillance” and “closely monitor.” Objective reporting this is not.

But the idea that surveillance is affecting academic performance is nuts. Tell me how that’s different than taking the same test in an exam hall with 200 other students and proctors walking up and down the aisles. Is that somehow not a “high degree of surveillance”?

To be fair, there have been studies showing a drop in test scores between proctored online exams and online exams that are not proctored. Usually, this is attributed to the proctoring limiting cheating, which artificially and unethically inflated the scores in the non-secure format.

But what does it matter? Afsharmehr does not offer any evidence of the claim anyway. It’s an assertion, dutifully and uncritically reported by the SMH.

And I have to add also — I am borderline screaming now — a flag of suspicious behavior does not mean anything. In more than 90% of cases, the student never knows there was a flag. It’s dismissed after a review, if it’s even reviewed at all.

Proving this point, there’s this:

When suspicious behaviour is flagged by the AI invigilator, the investigation process is often lengthy and labour-intensive. Ellis says human invigilators must then watch back the recorded footage and assess whether a student was likely to have cheated.

This can be difficult when seemingly innocuous behaviour – such as looking around the room or murmuring while pondering an exam question – is investigated.

“The university would get a flag, and then some human would have to sit and watch that video to figure out if that was cheating or thinking,” Ellis said of the process. “We have to pay a lot of human beings to do a lot of frankly quite boring work.”

Exactly. Proctoring flags are supposed to be reviewed by a human, ideally a trained human. And honestly, I’ve seen a good number of proctored exam videos and it’s not too hard to tell when someone is thinking and when someone is on their phone pulling up ChatGPT. Which means that the flag alone means nothing. There is a review. Innocuous behavior, as the SMH puts it, is dismissed.

But somehow, even though they reported this in the very same article, the logic escapes the paper, which reports that the flags are the problem.

It goes on:

Afsharmehr said he was concerned by the lack of transparency related to reviews by human invigilators. “They could really just think, ‘oh yeah, you’re looking at your phone, you’re cheating’,” he said.

Dude, lack of transparency? There’s a literal video that can be reviewed. It does not get more transparent than that. Not to mention, usually the cheating is pretty freaking obvious.

And of course the paper has a story of a student who is “really stressed” about the misconduct inquiry being conducted on one of his exams. Because it would not be news without the student who, and I quote, “maintains he never cheated.”

Continuing:

Ellis thinks the headache of trying to monitor student behaviour is a sign that universities should move on from exams altogether and assess progress more authentically, for example through one-on-one consultations with their professor.

So-called “authentic” assessment won’t derail cheating (see Issue 352). Although she is right that one-on-one conversations with a professor to assess learning mastery would be better than any online setting. No question. But I believe that’s wildly impractical.

For the record, the SMH story did not quote anyone saying that proctoring, while a pain, is necessary because of cheating. They did not include any of the several studies showing that exam proctoring reduces cheating. Three minutes on The Cheat Sheet and you could have found them. Here’s one, in Issue 169. But that would take work and the story is that remote exam monitoring is bad, so why bother to explore anything else?

Mostly, I just don’t know why we’re having this conversation, again. I did not understand why we had it the first time, honestly. Yes, remote proctoring sucks. No, you cannot give a credible online exam without it.

Nonetheless, while we’re on this securing remote exams is terrible and intrusive surveillance garbage again, I am delighted that I get to share one of my favorite academic integrity stories.

It is, on point, this article from the SMH. The headline is:

'You’re being watched and recorded, every breath': Students unsettled by exam software

It is essentially the same story the SMH ran this week — exam proctoring is bad because people are watching you take a test. It is funny that this story is from May 2020. Five years ago to the day that this most recent SMH article ran. Five years ago. Same misguided conversation. Same one-sided, hyperbolic reporting.

But that’s not what makes that 2020 story so amazing.

Please take two seconds and click that link to the 2020 SMH story. I’ll wait.

[he waits]

You’ll notice the photo of a young student sitting on her bed in her room. The caption is:

Emily Johnston is concerned about privacy after showing an anonymous exam supervisor around her room via webcam.

The article may be behind a paywall. If it is, here’s the image:

I hope you’re laughing just as I did five years ago.

You got it. This student invited a photographer into her bedroom to take a color photo of her, on her bed, to put in a newspaper with a paid circulation of 80,000 people and available online forever — to complain about the privacy of her room being violated. The story, with her photo, was on the front page.

Like I said at the time, and a million times since, it’s just hard to take this exam proctoring debate seriously.

And, yes, I remember this story and this photo from five years ago. I need a life.

Class Note

I’m going to take some time off from The Cheat Sheet this summer. A month, from July 15 to August 15, and return ‘round about back-to-school time.

I need a break, and I have things I want to get done. Academic integrity is my passion, but not my actual job.

I know, you’ll miss me. Please, keep yourself together. This is embarrassing.

I also have a plan.

Rather than have The Cheat Sheet go dark for a month, if possible, I’d like to keep The Cheat Sheet going while I’m on summer break. So, I’m asking for submissions I can share in the roughly ten Issues between July 15 and August 15.

If you have new research to publicize, new products to launch, new reports to promote, opinions you want folks to hear, please send them in. If it’s related to academic integrity or credential fraud, I’ll be pleased to share it.

I’ll do another little reminder or two as July 15 approaches. But please, if you’re interested and able, be thinking about sending in some articles. And thank you.