Futurism Goes Backwards, Adds Their Own Errors
Plus, GPTZero loves Course Hero. Plus, academic integrity conferences coming up in Europe. Plus, Honorlock to host webinar on AI detection.
Issue 217
To join the 3,326 smart people who subscribe to “The Cheat Sheet,” enter your e-mail address below. New Issues every Tuesday and Thursday. It’s free:
If you enjoy “The Cheat Sheet,” please consider joining the 13 amazing people who are chipping in a few bucks a month via Patreon. Or joining the 12 outstanding citizens who are now paid subscribers. Thank you!
Futurism Rehashes the Past, Incorrectly at That
Despite its name, the magazine Futurism decided to review and largely re-report the recent coverage by Rolling Stone of an academic integrity case at University of California, Davis (see Issue 215).
As addressed in 215, the Rolling Stone coverage was incomplete and infused with a few errors. And, as far as I can tell, the Futurism piece has nothing to add except for an error of its own.
The piece says:
The reality is that tools like Turnitin and GPTZero aren't very good at what they were designed to do. In our own testing earlier this year, we found that GPTZero fell far short of the mark. Even ChatGPT maker OpenAI's own detection app failed to reliably tell human-written from AI-generated text.
That’s the reality is it? Ridiculous.
One, they did not test Turnitin, or any of the other credible AI classifiers. Instead, they examined the two absolute worst products available - GPTZero and the OpenAI version. As I have shared many, many times, these two are terrible. In fact, earlier this month, in Issue 215, I wrote:
GPTZero is pretty awful. I’ve said this before and often (see Issue 191 or Issue 189 or Issue 187 or especially Issue 180). By my rough scorekeeping, it’s the second-worst AI classifier on the market. Only ChatGPT’s own detection tool is worse. If GPTZero gave me an assessment score, I’d get a second opinion.
But here’s Futurism, going back to their tests of GPTZero from January that showed - surprise, surprise - that it simply isn’t very good. Then they assume, based on nothing whatsoever, that all AI classifiers must be like those. And call that assumption “reality.”
Pretty amazing, if you ask me. Wrong. And pretty amazing.
Speaking of GPTZero
Though it has been abundant, so far, I’ve limited my criticism of GPTZero to its technology, which was cobbled together over a holiday break in a dorm room - at least according to media reports. For the most part, that has checked out. It was early to market, won a ton of press attention, and has quickly fallen into the lowest tier of AI detectors and classifiers.
In fact, GPTZero has been so star-spangled awesome at accurately detecting the hallmarks of AI content that, according to USA Today, the company is getting out of the business. Excuse me, pivoting:
[GPTZero Founder, Edward] Tian said GPTZero is pivoting from its former artificial intelligence detection model, and its next version will not be detecting AI "but highlighting what's most human."
Sure.
In the meantime, Tian’s visibility and inferior product have done real damage to the credibility of detection systems that are actually good.
With that background, I was stunned, though not at all surprised, when a reader sent me an e-mail saying that Tian, the founder of GPTZero, will be speaking at an event with cheating provider Course Hero later this month.
Here it is:
Somehow, the word “shame” does not quite cover this.
I’ll have more on the other speakers who are helping cheating profiteer Course Hero by appearing on their stage - as I’ve done before (see Issue 143 or Issue 44).
But back on Tian and GPTZero - it really is something to see the influence of cheating providers hijack those who have claimed to be on the side of integrity. GPTZero is chummy with Course Hero while CopyLeaks is in business with Chegg and Course Hero/QuillBot (see Issue 208).
Both CopyLeaks and GPTZero sell integrity products to schools. Getting paid by cheaters and by schools is a pretty sweet gig, I confess. If you can stomach it. Clearly, some can. And schools are content, it seems, to keep the money flowing - even in the face of some pretty headline-level conflicts.
True, I don’t know if Tian or GPTZero are being paid financially to shill for Course Hero, but they are standing in their spotlight, absorbing their visibility, tarnished as it is. For me, that’s plenty.
And should there be any question in the universe about what Course Hero is and what it does, see Issue 97. Or Issue 92. Or Issue 42 Or Issue 106. Or Issue 102. Or Issue 155. Or Issue 159. There is no chance that Tian does not know this. None.
Upcoming European Conferences on Academic Integrity in July and October
The European Network for Academic Integrity is hosting or co-hosting two conferences this year.
One is in July in Derby, England.
The other is in early October, in Galway, Ireland. The call for papers is open until June 19. This conference is in partnership with the University of Galway and the National Academic Integrity Network.
I wish I was going to either one. If you are, and want to tell us about it/them, drop me a note.
Honorlock to Hold Presentation on AI Writing and Detection
Academic integrity provider Honorlock will host a web event on AI writing and AI writing detection on June 29.
In the notice, they say that “many” people are:
realizing that trusted plagiarism checkers are ineffective at detecting AI-generated content.
Interesting.
Honorlock also says the presentation will feature:
real plagiarism scores from ChatGPT’s responses against 10 commonly used plagiarism detection tools.
Also very interesting. I may have to check that out.
I’ve asked Honorlock which ten checkers they plan to test or share; they have not responded. If they do before the 29th, I will let you know.