391: Open Season - The University of Cape Town Shuts Down AI Detection
Plus, the LSAT is over in China. Guess why. Plus, class notes.
Issue 391
Subscribe below to join 4,856 (+10) other smart people who get “The Cheat Sheet.” New Issues every Tuesday and Thursday.
The Cheat Sheet is free. Although, patronage through paid subscriptions is what makes this newsletter possible. Individual subscriptions start at $8 a month ($80 annual), and institutional or corporate subscriptions are $250 a year. You can also support The Cheat Sheet by giving through Patreon.
University in South Africa Doesn’t Want to Know About AI Use
According to reporting, The University of Cape Town turned off its AI detection technology, leaving it blind to who is using the technology and for what purpose — leaving students free to outsource their academic work with little chance of discovery.
From the coverage:
The University of Cape Town (UCT) has scrapped the use of artificial intelligence (AI) detection software as it shifts towards ethical AI use.
Speaking to Newzroom Afrika, director of UCT’s Centre for Innovation in Learning and Teaching, Sukaina Walji, explained that AI detection tools aren’t reliable enough to use confidently.
And:
“We decided to discontinue the use of the AI Turnitin Detector score for a number of reasons. The main reason is that, in our experience, these tools are not reliable enough,” she said.
And:
“What we are finding is that there are a lot of false positives, where students are accused of using AI when in fact they haven’t, but also false negatives,” said Walji.
“These are just not reliable tools for us to feel comfortable to continue using them in the teaching and learning ecosystem.”
It’s clear that the university is not — was not — using AI detection technology correctly. If you’re discussing “reliability,” you are missing the purpose and function of these tools.
Then there’s this:
“Our approach is two-fold. One is to continue talking to students about why they should think about using these tools in their assessments because it’s really about the learning,” she said.
“Also, we have to think and change our assessments. AI is here. Students are using artificial intelligence tools, and so assessment strategies need to change.”
Good luck with that.
No chance.
Instead of using metal detectors at the airport, The University of Cape Town is going to remind people that hijacking a plane is bad because it’s really all about getting to your destination. Sorry, they won’t even say that hijacking is bad, just that people should “think about” what they do when they get on an airplane.
That’s A+ stuff right there.
There’s also this:
“I think what is happening is that the assessment practices themselves need to change and are changing. It is the role of the university to assess students with integrity.”
Letting people act dishonestly, for any reason, is not congruous with integrity. You cannot know that fraud and unfairness are happening and say nothing, while doing even less. You cannot, in my view, have the power to demonstrate an intolerance to illicit, damaging behavior and decide to look away. Well, you can. You shouldn’t. And if you decide to simply look away, you absolutely should not talk about doing anything “with integrity.”
Another View
After the above, the news outlet that carried the story, MyBroadband, does something pretty cool. They offer up, in the same article, an “Alternative view.” The one in this story is from:
Andy Carolin, an associate professor at the University of Johannesburg’s English department
Carolin:
says students’ use of AI poses a significant threat to the credibility of their degrees.
You don’t say.
He continues:
“I think that we’re much closer to a crisis than many of us are willing to acknowledge, and it’s worth pointing out that this is not fear-mongering,” he said.
“I have no doubt students use ChatGPT to submit written work, but what sets the large language model apart from others is that there is just no way to prove it.”
I think he’s right about the crisis. He may also be right about proving AI use. I don’t know that I’d say it’s impossible to prove, but I’d say that proof misses the point, or that it’s the wrong point. Absent an admission, almost no cheating is possible to prove. Catching a student with a cheat sheet in their hand, during a test — sure. Otherwise, absolute proof is not the standard. It has not been for a long time, if it ever was.
Even so, the bigger point deserves the fidelity of our focus — cheating is destroying the credibility of academic attainment.
The University of Cape Town, it seems, doesn’t care. Or at least they don’t care enough to use available tools to try to limit cheating. Unplugging, literally and metaphorically, is just easier. If ignorance is bliss, The University of Cape Town is downright giddy.
I’m trying to rant less. It’s not working.
Moving ahead with Professor Carolin’s response:
Asked about the effectiveness of AI detectors like Grammarly or AI Turnitin Detector, Carolin said they only provide a surface claim of AI use.
However, he said this isn’t the only way in which AI affects tertiary education. He warned that many students don’t even attempt to engage with prescribed course material.
As a result, he said, many students are graduating with degrees, having never been thoroughly assessed or taught critical thinking.
“We risk an ever-increasing number of students who hold certificates that fraudulently certify their mastery of skills and content knowledge that some may have only barely attempted.”
I don’t have much to add.
Yes, AI detection gives you a surface claim of AI use. They are like metal detectors on the beach. They tell you where to dig. I’ll never understand why this is complicated. My premise is that, for some folks, not understanding is just easier. You never find what you don’t look for, and if you don’t find it, you cannot be expected to do much else.
But the last quote, above, is the point.
We’ve reached the time, the fork in the road, where schools and academic leaders are going to be called on, and counted, in this crisis. Ignorance or confusion is no longer an acceptable response. We’re past that.
Some are going to put in the time, work, and money to protect and reward integrity, to invest in the value of learning and the value of the degrees they award — others won’t. Very soon, if it’s not already, it’s going to be painfully easy to tell one from the other.
LSAT Suspended in China
A friend of The Cheat Sheet sent us this important development — delivery of the LSAT, the Law School Admissions Test — has been suspended in China.
Go ahead, guess why.
According to the announcement from the test provider:
We have been increasingly concerned about organized efforts by individuals and companies in mainland China to promote test misconduct.
They continue:
While security is always a concern, these enterprises are becoming increasingly aggressive.
Yup.
I don’t mean to single out China. It’s one of a handful of countries in which test fraud is incredibly common and incredibly profitable. It’s so bad that any test delivered online in China is, in my view, compromised beyond validity.
To be clear as well, this is not a new problem (see Issue 232). In Issue 137, we noted that organized criminal gangs in India were giving up selling drugs because selling test fraud was more profitable.
More from the announcement:
This type of [cheating] activity is not limited to the LSAT; these enterprises purport to offer cheating services for virtually every standardized test.
True. Again — this is not a China problem or an LSAT problem. But this is a gigantic problem.
The announcement again:
After careful consideration, we have decided to take the additional step of suspending online testing in mainland China following the upcoming October international administration of the LSAT. We will be taking a variety of steps to enhance the security of the October LSAT. Because we do not currently offer in-person testing in China, the October test will be the last LSAT administration in mainland China until further notice.
And — round of applause.
This was not an easy decision. The LSAT in China must be a cash machine. Pulling it off the shelves involves more than just money, it raises real questions of fairness and access. So, seeing a company put the validity of their assessment and the sanctity of its scores ahead of money and ahead of awkward questions, is great.
It’s great.
If people keep stealing your lunch money, quit carrying your lunch money until you can figure out a better way. Like this:
We will continue to monitor and respond to this situation and will continue to evolve our security measures and employ a wide range of tools to protect the integrity of the test both in the U.S. and internationally.
Integrity is not cheap. But it is worth more than whatever it costs. Good for LSAC, the test provider.
And I know this is crazy, but every standardized test ought to hold themselves to the same standard. Give a secure, valid assessment or don’t give one at all. Colleges and universities, I’m looking at you.
Anyway, this is big news, and I do hope that others recognize the leadership this takes.
Class Notes:
I’m back. Sorry. After a few weeks off, I’ve returned. Thank you to those of you who sent in material to share during my hiatus. Cutting and pasting is so much easier than actually writing — cough. Also, I won’t catch up to the news. I was behind when I took the break. But if you think I skipped over something important, let me know.
My return does not mean submissions are closed. I’ve always been happy to share news, notes, views and other voices here. If you have something related to cheating or integrity to share, and you think this is a good place for it, send it in.
Finally, I’m happy to share that I’ll be presenting at the 2025 Conference of OLC, the Online Learning Consortium. That’s in Orlando in November. My topic: Are Online Assessment Securable? Accordingly, the LSAT news above seems prophetic.