On ChatGPT

Plus, an article on more new research in Australia. Plus, International Quick Bites.

Dec 08, 2022

Issue 170

To join the 2,620 smart people who subscribe to “The Cheat Sheet,” enter your e-mail address below:

If you enjoy “The Cheat Sheet,” please consider joining the nine amazing people who are chipping in a few bucks a month via Patreon. And thank you to those who are.

Support "The Cheat Sheet"

A few things about ChatGPT, AI writing and Academic Integrity

With the obvious academic integrity implications of ChatGPT - the new conversational AI writing platform - my inbox cluttered up a bit.

It will surprise no one that I have a few thoughts.

Just the Facts

One, I don’t think ChatGPT is the global threat to writing assessment that most people assume it is - at least not yet. That’s because, while it’s mind-blowingly good at writing and its flexibility is other-worldly, it makes stuff up. At least for now, ChatGPT is struggling to separate fact from fiction. It does not source well, or at all.

I don’t know if that is fixable. But at least for the time being, instructors ought to be able to spot obvious factual errors or invented attribution or suspect sourcing in written course and assessment work. And while that may not point directly to having used AI, it will point directly to a bad grade.

Ironically, AI may not cover a learning deficiency, it may outright expose it.

And word will get around that the AI makes errors and students will be cautious. After all, if they don’t know the material, they won’t know the AI is wrong. They will be sitting ducks.

But again, that may be a temporary fix. If ChatGPT gets its facts straight, that will be different.

Essay Mills

A few very smart people either wrote to me or commented publicly that ChatGPT may be the end of essay mills. I don’t think so. In fact, I think it will likely make them more popular and more profitable.

To start, ChatGPT can significantly reduce an essay mill’s time and investment per contracted piece, increasing its reach and profitability. It’s well known that essay mills recycle their papers, at least as templates. Imagine if they didn’t even have to do that but could start out with 1,000 words of pretty good text on any topic in seconds.

I am sure essay mills are already using ChatGPT to cut their overhead.

Next, if accuracy turns out to be an Achilles heel for ChatGPT, having an essay mill play fact-checker and cleaner will be highly valuable.

What if, for example, essay mills went from a turnaround time of days to just hours and shifted their labor to fact-checking and sourcing? If they just cleaned up what ChatGPT spit out?

That may be a different model for selling cheating but my guess is it will be better, faster and even more profitable.

Plagiarism Detection

I saw someone post somewhere that GPT-generated text was getting perfect scores from Turnitin - indicating it was not plagiarized. Their point was that ChatGPT was going to render Turnitin ineffective because the “writing” was “original” and undetectable.

I disagree here too because people who know Turnitin know that a score of zero is just as suspicious as a score of 80. Facts and citations, if nothing else, get repeated. A score of zero means the paper being assessed is likely complete fiction. New, but fantasy.

Turnitin and schools may have to - probably should - invest in educating educators about what their scores mean again. But if it’s true that ChatGPT is getting no flags from Turnitin, that’s another easy way to spot AI-written material.

Policy Change - Retain, Retain, Retain

Unrelated to the above, schools should immediately promulgate a policy of long-term retention for all written course work. Like ten years or more. A simple policy that says, “you turn it in and we grade it, we keep it for ten years.”

Put it in the syllabus and make students agree to it.

The reason is that, while technology is not yet up to the task of detecting good AI-written work with accuracy, and especially not that of ChatGPT, it’s a fool’s bet to assume it will never be.

By keeping written work a long time, it raises the prospect that when the technology catches up and it can pinpoint ChatGPT work, inquires and penalties may be in order. Even after graduation. In other words, a student may use ChatGPT to fool a teacher today, but they’d also have to bet that ChatGPT will be good enough to fool the detection technology of nine years from now.

Such a policy would help change the stakes of using AI and offer another possible disincentive to cheating. And at very little cost.

Have thoughts or questions or opinions on academic integrity and ChatGPT? E-mail me. Though I may not be able to answer the questions, I will see what I can do. A reply e-mail gets to me.

Researcher: Because of Cheating, Schools Should Revert to In-Person Exams

Although I am loathe to share much of anything from The Conversation (see Issue 166), Meena Jha, a researcher with CQUniversity Australia has a piece there on cheating that’s worth passing around.

It starts off right on point:

Contract cheating – where commercial cheating services provide assignments for university students – has become a global problem.

And gets right to this news item about the University of New South Wales which:

said it was detecting more than double the amount of cheating among its students post COVID. Before the pandemic, just under 2% of students were caught in misconduct processes each year. Now it is close to 4.5%.

Jha doesn’t hold back either about why all this matters:

This isn’t just problem for individual universities. It threatens the integrity and reputation of a university degree and the whole higher education system.

And while I dissent from her view on penalties and deterrents, she does correctly cite that:

Research in 2021 showed one in ten students either pay someone to write their essays or use content they find that was not written by them. Other studies show up to 95% of cases go undetected.

Yes, 95%.

But the news is that she has new research. According to her piece in The Conversation, she and her team interviewed “academics” from 41 Australian universities on the topic of academic integrity.

On switching to online exams, she writes:

As one interviewee told us,

There was a lot more cheating, both plagiarism and collusion […] students are cheating in way that they were not able to cheat with paper, supervised exams.

Another explained:

we would release the exam at 8am […] and about 20 minutes later the questions were appearing on the contract cheating sites […] we did think of limiting the time they had available to do the exam, but clearly, the internet moves faster than we do.

Contract cheating sites. Hmmm. She added:

Another academic was more blunt:

you cannot ensure academic integrity in online assessment.

She explores the interesting idea of adding oral reviews or assessments after online exams but says her interviewees told her not even that stopped the cheating. So, she suggests going back to:

traditional face-to-face exams, with student identity card checks, arranged seating, and exam rooms monitored by staff.

To be clear, that won’t stop cheating. But it remains exceptionally clear that establishing any baseline of academic integrity in online exams - anything on par with those in physical classrooms or test centers - is a challenge.

One hundred schools in Kenya were put on “high alert” or “red alert” recently over “possible exam cheating.” Authorities promised to crack down on cheating and punish cheaters.
In India, a man was arrested after he tried to impersonate a test-taker for a national forestry exam. And apparently, that’s the plot of a Bollywood movie.
An “elite” private school in Brisbane Australia apparently had a cheating scandal. The principal denies students escaped punishment. That’s all I know because that’s all I can read without buying a subscription.