Here We Go Again - More Really Bad Press Work on Academic Integrity
Plus, Inside Higher Ed and 72 hours. Plus, an interesting job opportunity at Grammarly.
Issue 316
Subscribe below to join 4,149 other smart people who get “The Cheat Sheet.” New Issues every Tuesday and Thursday.
If you enjoy “The Cheat Sheet,” please consider joining the 16 amazing people who are chipping in a few bucks via Patreon. Or joining the 41 outstanding citizens who are now paid subscribers. Paid subscriptions start at $8 a month. Thank you!
EdWeek Really Blows Coverage on K-12 Use of Generative AI — Really Blows It
I’m sorry — I am annoyed.
Reporting has consequences and really bad reporting has really bad consequences. When an outlet reports on research, most people — by far — read the coverage of the report instead of the report. Likewise, by far, most social sharing of the research, and most headline grabbing, also comes from the reporting, not the report.
So, when Education Week (EdWeek) gets its reporting wrong at a fundamental level, as it did here last month, most people get the wrong information and make wrong conclusions.
We’ve seen this before, in this absolutely horrid “research” from Stanford on a supposed bias of AI detectors against non-native speakers (see Issue 216). It’s been repeated and shared in press accounts as fact. And every time I think about it, I remain stunned that no one seems to care that the paper has footnotes that appear to be complete invention. However, someone wrote up how AI detection has a bias, so it must be true.
EdWeek Reports Fiction
Which brings us back to EdWeek. The story is about a paper from Common Sense Media, a survey of teens and their parents about AI use. The EdWeek story seems to have a dead link — the link to the underlying paper is here.
The survey sample is robust and the data collection entity — Ipsos — is credible. It oversampled Black and Latino families, which is good. No issues with their data or methods. Or even their conclusions.
But to understand where EdWeek went off the rails, we can start with their headline:
Black Students Are More Likely to Be Falsely Accused of Using AI to Cheat
That would be quite the finding if it was accurate. But, of course, it’s not.
For one, that attention-getting headline is not the report’s title given to it by its authors, the ones who actually did the research. The report title is, “The Dawn of the AI Era: Teens, Parents, and the Adoption of Generative AI at Home and School.” The data that EdWeek used for their headline is not even mentioned in the introduction. It does not come up in the report until finding number seven — 7! — on page 15. In the report’s conclusion, it is not mentioned until the middle of paragraph seven.
That’s pretty precise cherry-picking.
I mean, this specific data point about Black students and false accusations was so critical to the authors of the actual report they literally did not mention it at all in their press release. But to EdWeek, it is the story.
Still, that’s minor in comparison to the fact that the report itself did not say that “black students are more likely to be falsely accused of using AI to cheat” — except for in one place, more or less, which I believe to be a writing or editing error.
As mentioned, the report is on a survey and the authors are very particular in phrasing the findings, like these four examples for this particular data point:
Compared to their White and Latino peers, Black teens are about twice as likely to report that teachers flagged their schoolwork as being created by generative AI when it was not.
And:
Black teens are about twice as likely as their peers to report that teachers flagged their schoolwork as being created by generative AI when it was not.
And:
Black teens are twice as likely as White and Latino teens to report that their teachers wrongly flagged their schoolwork as being produced by generative AI when that was not the case.
And:
Black teens are more than twice as likely as White or Latino teens to say that teachers flagged their schoolwork as being created by generative AI when it was not (20% vs. 7% and 10%, respectively).
Note that in all four sections there is some version of “to say” or “to report.” That’s right — since it is a survey, the report can only say that Black teens said their work was flagged improperly. It does not mean it was flagged improperly. There is no way for the research paper to know that and, to its credit, it never says otherwise. Except, for this example, which is the one I believe to be just a copy error, since the report uses the more precise language in many other places:
Black teens are most likely to have schoolwork incorrectly flagged as created by generative AI
There are actually two errors there, which, again, makes me think it’s just a misfire.
But EdWeek went with the definitive, even though it’s not there — saying as a matter of fact that “Black students are more likely to be accused.” And before you think that such an unsupported jump may be a simple causality of headline inaccuracy (which can happen), here is the full first paragraph of EdWeek’s reporting:
Black students are more than twice as likely as their white or Hispanic peers to have their writing incorrectly flagged as the work of artificial intelligence tools, concludes a report released Sept. 18 by Common Sense Media, a nonprofit that examines the impact of technology on young people.
Wrong. Just wrong.
Again, the students said the flag was wrong, which does not mean it was. Stating what should be obvious, there are reasons students may want to say, or may in fact believe, that a flag of potential AI text was wrong. When people accused of cheating say they were not cheating, it does not mean they were not cheating.
When I say I am Napoleon Bonaparte, you cannot report that I am Napoleon Bonaparte. All you can say, at best, is that I say I am. Well, I mean, that’s true if you’re doing actual journalism.
How is this complicated?
On Accusations
While we are here, back to the EdWeek Headline. It says Black students “are more likely to be falsely accused of using AI to cheat.” The paper itself never even uses the words “accuse” or “accusation.” Not one time — but the loaded word makes its way into EdWeek’s headline.
Later, EdWeek tells readers:
But 20 percent of Black teens were falsely accused of using AI to complete an assignment, compared with 7 percent of white and 10 percent of Latino teens.
To beat this horse more, the paper does not mention accusations at all. The research is completely silent on the topic. Since a flag of potential, or even suspected AI use, does not always result in an accusation, the research was right to avoid it. It only asked about work being flagged. Different.
If an example helps, “Hey Derek, this AI detection thing flagged your paper as possible AI. Since I know you and your writing style, since we worked on it together in class, I know the detector is wrong. Anyway, funny story.” Incorrect flag: yes. Accusation of cheating: no.
But, (slightly shouting now) neither the report nor the EdWeek writer has any information as to whether these supposed AI flags were, in fact, false. Zip.
Bias About Software Bias
Convinced of their conjured conclusion that Black students are “falsely accused” more, EdWeek goes on and blames the AI detection software:
This may be at least partially due to flaws in AI detection software.
And:
AI detection software has already been shown to have problematic biases
To support the second sentence, EdWeek uses that bogus Stanford study I referenced earlier. The one that has more red flags than a matador convention.
In the first sentence above, the word “partially” is asked to do some pretty heavy lifting. At least in this regard, there is some scant foundation in the actual report, which says about the point EdWeek wants to make:
This suggests that software to detect AI, as well as teachers' use of it, may be exacerbating existing discipline disparities among historically marginalized groups
Still, what is said in the original report and what is said in the EdWeek piece are subtly different, which is very important. The report is more careful and more accurate in that, because AI detection systems are tools, how the tool is used is crucial. When bias shows up, it’s sure to include the instructor, at a minimum. The report is right to use “as well as.”
And although EdWeek wrote first about possible “flaws in AI detection software,” they say later in the story:
Common Sense Media’s findings on Black students could be due to either unfairness in AI detection tools or biases in educators themselves, according to experts.
Ah, yeah. Exactly. The experts say the bias is either/or. It could be the teacher entirely. But EdWeek is on the software, writing about “flaws” and walking us through the terrible Stanford research again. Those are choices.
What the Report Does Say
I think it’s interesting and rather telling that there were also integrity-related findings in the report that EdWeek chose not to share.
Here’s one:
Teens say they are using generative AI for a variety of purposes, but using it for help with homework is the most common.
Help with homework is most common? You don’t say. EdWeek didn’t.
Moreover, the report says that the plurality of students:
46%, used gen AI for the assignment without the teacher's permission.
So, I guess we’d call that cheating.
Also of the those students who said they used AI on a school assignment:
63% have used chatbots or text generators
Text generators.
Stringing all that together, the most common use of generative AI among teens, according to the survey, was for homework. Most of that group used text generators and the largest portion did so without the teacher’s permission.
EdWeek did not mention any of it.
Also unreported by EdWeek was that the survey asked teens to agree/disagree with some statements about “gen AI in schools.” The top answer by agree rate was:
Gen AI could be used to cheat in school 77%
Huh.
The survey also asked parents the same agree/disagree battery. The highest agreement rate among parents? Wait for it:
Gen AI could be used to cheat in school
Seventy-eight percent.
EdWeek: crickets.
Not Perfect
The Common Sense report is not perfect.
In addition to that one fumble conflating what people said was happening with what was actually happening — something they got right a half dozen other times — the report kind of bizarrely says:
Recent studies have suggested that initial fears about widespread increases in cheating may have been overblown
That’s a reference to the research we covered in Issue 261. To be fair, that is what that report says. But as I wrote at the time, I am not sure that is the real finding:
If your data shows 60-70% of students admit to cheating “recently,” and we know that survey data is a low marker of actual misconduct — you may already be showing a real cheating rate at or near 100%. If the research on actual cheating is 2.5x what self-surveys show, the folks at Stanford have bumped their heads on the ceiling. There is simply no room in the data to move upwards.
Let me try this another way — using AI to cheat will not make high school students more truthful about cheating.
But that’s detail. And skepticism — two things EdWeek seems to have lost in transit on this occasion. And accuracy. Three things.
It matters because I’ve seen many people — you know who you are — sharing this EdWeek story as proof that AI detectors are racially biased and inaccurate.
Unfortunately that that is what EdWeek said. The problem is, as I hope I have pointed out, there is just no way to substantiate it — it is profoundly misreported. To the point of fiction. And inexcusable.
Inside Higher Ed, Three Days Later
Last week, regulators in Australia sued Chegg for violating the country’s anti-cheating laws (see Issue 314).
As mentioned in our write-up, UK-based Times Higher Ed ran the story. That was on October 8 — Tuesday. Even though they have the same owners, Inside Higher Ed, based in the US, ran the Times story three days later — On Friday.
I’ve pointed out many times how bad Inside Higher Ed (IHE) has been on issues of academic honesty, not just promoting cheating companies, but acting as as an actual source of disinformation on the topic. I won’t rant too hard on this since you can check the tape yourself — see Issue 274, or 195, or 311, or 167, or 102, or 53, or 49, or 50.
I did not include even half the examples.
Further, in putting this little article together, I noticed two other IHE examples I’d missed — both promoting Chegg:
An interview with a PhD student, featuring her work as “a researcher with Chegg’s Center for Digital Learning.”
An article on learning AI skills featuring a survey from Chegg.
Both treat Chegg as if it’s a credible company, a credible source. Neither mentions the company’s affiliation with cheating.
Back on point — frankly, I am surprised IHE ran the Chegg/Australia story at all, even though Chegg is an American company and most of their customers are in the US. But I do rub my chin a little bit over the three-day delay, especially since what did run was a reprint of the story from Times Higher Ed.
Whatever. I guess I am just glad they ran anything at all that’s critical of cheating or a cheating provider.
Although IHE still has not, as far as I can tell, covered the story about Chegg’s investor fraud lawsuit and that a judge ruled that the company likely profited from cheating and likely misled the public about it (see Issue 280). That happened in March. It’s October.
Eye Raising Job Opening
Grammarly, which has gone from helpful grammar assistant to a complete AI writing tool — doing the writing for you — (see Issue 315) is a big company and, as such, has many open positions. One caught my eye.
Grammarly is seeking a “Sales Engineer, Grammarly for Education.” It’s a sales job that “will support Account Executives on business opportunities with Educational institutions.” The company wants someone who, “has previous experience selling to higher education.”
That’s all fine. But two things in the job post raised my eyebrow.
One, Grammarly describes itself as:
the world’s leading AI writing assistance company trusted by over 30 million people and 70,000 teams. From instantly creating a first draft to perfecting every message
Yes, Grammarly instantly creates first drafts of written work now. That is their business, in addition to helping you wash and clean up that work and avoid plagiarism detection.
Furthermore, this job post in particular says that the higher education sales leader Grammarly wants to hire:
will be crucial in expanding the use of our innovative, trustworthy AI writing tools.
Again, AI writing tools.
Grammarly wants to expand the use of AI writing tools in higher education. They want to sell more AI writing to and in colleges and universities.
Good for them.
However, I argue that expanding the use of AI writing tools in education, especially for profit, is a real threat to — or at least a magnificent temptation to circumvent — integrity. When students use Grammarly to commit academic fraud, as they are, the company will have to choose between profit and integrity. I’d like to be wrong about this, but it’s pretty clear to me already that Grammarly has chosen the cash (see Issue 282).