The 74 Gives a Strong Nominee for Worst Piece of Integrity-Related Journalism
Plus, hack an exam, pick up a grand. Plus, an update from ASU.
Issue 295
Subscribe below to join 3,862 other smart people who get “The Cheat Sheet.” New Issues every Tuesday and Thursday.
If you enjoy “The Cheat Sheet,” please consider joining the 18 amazing people who are chipping in a few bucks via Patreon. Or joining the 38 outstanding citizens who are now paid subscribers. Paid subscriptions start at $8 a month. Thank you!
Avert Your Eyes - The 74 Writes That “Robocops are Watching Your Kids Take Online Exams”
I thought we were past all this. I really did.
But here we go.
Again. Like we haven’t had to live though nine hundred of these stories already.
Nonetheless, last month, folks at The 74 decided to publish a story on online exams decrying the use of remote proctors for online assessments. Again. And, as you may be able to tell already, it’s awful — as in, completely segregated from reality and overtly biased.
It starts, as these stories tend to, with this ominous intro:
In the middle of night, students at Utah’s Kings Peak High School are wide awake — taking mandatory exams.
Curiously left until later is the fact that this is an online school, and that while the exams are mandatory, when the students take those exams is up to the student. No one is making students take tests “in the middle of the night.” If students choose to take exams during non-traditional hours, the school will use an online security feature to dissuade cheating. It is a clear and reasonable trade off.
Also buried in the article is that, at King’s Peak:
anybody with privacy concerns is allowed to take their tests in person.
So, the students are literally opting in on remote proctoring in order to be able to take exams when they want to.
The horrors of the surveillance state know no bounds.
But before we even get there, the story should be over. In fact, it’s really no story at all because, underlying all this, beneath all the hyperventilation and hype, is that this is very simple — education providers have obvious and unavoidable choices to make. They are:
Don’t offer online programs or classes.
Offer online programs but don’t measure anything, don’t give assessments.
Go online and measure outcomes, but don’t deploy cheating safeguards.
Have online programs, give assessments, and use anti-cheating systems.
That’s it.
Those who clutch their pearls about using integrity and security systems in online learning should, in my view, be required to specify the alternative they prefer.
Of course, The 74 doesn’t. But I suspect someone at The 74 is in on the joke because the story says:
Any student who feels compelled to cheat while their teacher is sound asleep, however, should know they’re still being watched.
“Still” is the key word.
If students took these same exams in a physical school building, they’d be watched. If they took the same exams during regular class hours, even online, they’d also likely be observed. In other words, setting a camera or automated system to review an exam taken at odd or irregular hours is a distinction with no difference — a student will “still” be watched in every case.
Nonetheless, immediately after conceding that students are being watched no matter the setting, The 74 goes off the deep end of sanity with this:
For students, the cost of round-the-clock convenience is their privacy.
That’s bananas. Cost is clear — something is being paid for convenience. The 74 thinks it’s privacy. Even if that’s the case, it’s a cost many students are obviously happy to pay. And personally, I think you’re already and involuntarily sacrificing privacy in a classroom of 40 under the eye of a teacher.
Following the ‘cost is privacy’ thing, the piece continues immediately:
During exams, their every movement is captured on their computer’s webcam and scrutinized by Proctorio, a surveillance company that uses artificial intelligence. Proctorio software conducts “desk scans” in a bid to catch test-takers who turn to “unauthorized resources,” “face detection” technology to ensure there isn’t anybody else in the room to help and “gaze detection” to spot anybody “looking away from the screen for an extended period of time.”
Proctorio then provides visual and audio records to Kings Peak teachers with the algorithm calling particular attention to pupils whose behaviors during the test flagged them as possibly engaging in academic dishonesty.
To be as clear as I know how to be — aside from recording it, nothing in this section is different from what would happen in a regular, in-person class. Desk scans, yes. Face detection, yes. Gaze detection, yes. Knowing where a student is looking, which is kinda the same thing as gaze detection, yes.
The singular difference aside from the recording is that when the test is being delivered by a person, a person does the observing, and when the test is delivered online and by computer, the observing is done by computer. One, we accept. The other is Armageddon.
Citing the use of remote proctoring services at colleges and universities, The 74 writes that the practice:
met with sharp resistance from students. Online petitions demanded institutions end the surveillance regime; lawsuits accused the tools of violating their constitutional rights and relying on “racist algorithms” that set off a red flag when the tool failed to detect Black students’ faces.
First, “surveillance regime.” Come on. At least pretend to be objective.
Sure, many students hated efforts to make it harder to cheat. I never doubted that students prefer easy paths to good grades. And students did make those accusations — and more. But accusations are not facts, and the result of all that protestation, as far as I know, has been next to nothing. In higher education settings, remote proctoring is the norm because, again, the alternatives are unjustifiable.
But The 74 has blown apart the case with:
an analysis by The 74 has revealed that K-12 schools nationwide — and online-only programs in particular — continue to use tools from digital proctoring companies on students, including those as young as kindergarten.
I don’t have much to say on the kindergarten thing. It feels creepy and I’d like to know the circumstances, which the publication does not provide. But yes, many schools — and especially those that are online — use digital proctoring to ensure the fairness and validity of assessments.
Bombshell.
The true scandal would be if they did not.
Speaking of, the article does also say that an:
August 2023 educator poll, [found that] 36% of teachers reported that their school uses the surveillance software.
Surveillance software. But, if the survey is valid at all, it implies that nearly two-thirds of schools don’t monitor their online exams? Do two-thirds of schools even have online exams? Not sure. Something is probably off with applying that finding. Even so, I’d rather answer a question about why my school records and reviews online exam sessions than try to explain why it does not.
The piece quotes Chad Marlow, a senior policy counsel at the ACLU, who I can say with conviction has zero idea what he is talking about. Marlow, says:
It’s the same theme we always come back to with student surveillance: It’s not an effective tool for what it’s being claimed to be effective for
Wrong. He’s just wrong. The research is overwhelming that remote assessment proctoring reduces cheating. But you won’t find any of that in the article. More on this in a moment.
Further in, the article examines a casual test by a college student, claiming that Proctorio’s facial detection software was biased. At the time, the company correctly argued that the data used to test Proctorio’s system contained images such as cartoons, toddlers, and images with multiple faces in the same image — things that should rightly be flagged as problematic in a test setting. Here’s the image Proctorio shared of examples of the ‘faces’ used to accuse the company:
It is pretty nutty that this data was used to make accusations of racial bias. Nuttier still is that some people believed it.
But what’s nuttier yet is that The 74 quotes former library staffer Ian Linkletter to comment on these supposed findings. Linkeletter has initiated — and lost — a series of lawsuits against Proctorio (see Issue 267). Quoting a frequent litigant with no particular expertise in facial detection technology or academic integrity is a choice.
Continuing, The 74 decides to go after whether remote test proctoring works:
While there is little independent research on the efficacy of any remote proctoring tools in preventing cheating, one 2021 study found that Proctorio failed to detect test-takers who had been instructed to cheat. Researchers concluded the software is “best compared to taking a placebo: It has some positive influence, not because it works but because people believe that it works, or that it might work.”
As mentioned up top, this is false. It’s so blatantly false that it must be either overt bias in omission of facts, or simple ignorance. Had The 74 interviewed a single source with a passing understanding of academic integrity, the errors may have been corrected. But they didn’t — at least they did not quote any.
First, I am aware of at least seven independent academic studies demonstrating the efficacy of remote proctoring. Seven. In just the last four years. If I have time, I’ll try to list and link them in a future Issue just to show how not hard it is to find this information.
On the paper the article does cite, it’s an absolute sham. I gave it a dishonorable mention for the worst academic “research” of 2022 and covered it deeply in Issue 153. I am loathe to rehash it but let me quickly share:
The research sample was six. Just six.
The six students were not actually cheating.
The six students volunteered, and were all computer science majors.
The six students met and planned their deceptions in advance.
Moreover — the same study — found:
students are (in a clear majority) of the opinion that the use of Proctorio will prevent cheating.
And, citing existing research on the effectiveness of remote proctoring — the same study — said:
it confirms that the use of online proctoring has a preventive effect, as was also suggested in our own student survey
So, proctoring works to prevent cheating. But that’s just one very bad study. Seven others show the same thing, only better. The 74 doesn’t share any of those. Again, a choice.
In another choice, The 74 writes:
Documents indicate that K-12 institutions continue to subject remote learners to room scans even after a federal judge ruled a university’s use of the practice was unconstitutional. In 2022, a federal judge sided with a Cleveland State University student, who alleged that a room scan taken before an online exam at the Ohio institution violated his Fourth Amendment rights against unreasonable searches and seizures. The judge ruled that the scan was “unreasonable,” adding that “room scans go where people otherwise would not, at least not without a warrant or an invitation.”
While factually accurate by the width of a needle, this is wildly incomplete. For example, the ruling applied to that single student only. The Judge found that the school was wrong as it scanned that single student’s room that one time, after a series of serious mistakes. The Judge did not ban room scans and they were not declared unconstitutional. It’s also important to note that in this Ohio case, the school did not use a proctoring company. It used Zoom. See Issue SE 2 or Issue 188 for more.
Nonetheless, The 74 gives the impression that they are banned. There’s just no other way to read the phrase, “institutions continue to subject remote learners to room scans even after a federal judge ruled…”.
Whatever. I am irritated. And I expected more from The 74, honestly. I know it takes a ton of work to be this bad. The effort could have — should have — been put to better use.
Hack an Exam, Get $1,000
The good folks at assessment security provider VICTVS have launched an exam hack event, enlisting and encouraging people to try to hack their exam platforms. The winner, the company, says can score $1,000.
This is from VICTVS:
Have you ever cheated in an exam? Do you despise exam cheaters? Do you want to try and win $1000?
If you answered YES to any of the questions above, then it's time to sign up for the Global VICTVS Exam Hack 2024!
VICTVS are looking for students and academics worldwide who are up for the challenge of beating our industry leading exam software and invigilators.
Sit an online test, and see if you can cheat your way to victory. You'll have to come up with creative, elaborate or strategic ways to cheat! The most innovative and interesting hack wins a cash prize of $1000!
Ready to take on the challenge?
To learn more and register your interest, visit The Global VICTVS Exam Hack 2024 website now : The Global VICTVS Exam Hack 2024
I love this. One, it’s brazen. Two, it’s great for research and product development. Three, I’ve always thought schools should build and offer a degree in cheating.
You may think I’m crazy, but bypassing exam and assessment security is a skill. It requires planning, creative thinking, and goal-oriented action. If a student can cheat their way through a designed course of assessments, that shows me something. Schools can’t award degrees in the assessment content or subject matter, obviously. But I can think of a few hiring entities that might be interested in someone with a demonstrated propensity for sidestepping security regimes.
Just an idea.
Anyway, the VICTVS thing sounds like fun. If that’s your kind of thing, dive in.
Reminder: if you are in the academic integrity space — assessment security, course design, schools or school integrity offices, testing centers, academics — and you have news or events to share with the academic integrity community, please send them in. I am happy to spread the word and there is no charge.
Update on ASU Online Computer Science Class Cheating
In the last Issue, we dug into cheating at Arizona State University and the remarkable memo that a professor wrote about the incident.
The short update is that the memo is real. An educator and leader of the course in question confirmed it. The school itself has not responded.