Honorlock: Leaked Exam Questions Have Tripled, More than One-Third are Compromised
Plus, get your popcorn: Course Hero's offensive anti-proctoring panel. Plus, a follow-up from the professor who posted fake answers on Quizlet.
Issue 219
To join the 3,337 smart people who subscribe to “The Cheat Sheet,” enter your e-mail address below. New Issues every Tuesday and Thursday. It’s free:
If you enjoy “The Cheat Sheet,” please consider joining the 13 amazing people who are chipping in a few bucks a month via Patreon. Or joining the 13 outstanding citizens who are now paid subscribers. Thank you!
Honorlock Shares Cheating Stats
As mentioned in Issue 217, Honorlock hosted a webinar on AI cheating and AI detection this week. I was able to drop in for most of it.
Surprisingly, Jordan Adair, Honorlock’s VP of Product, shared some statistics on cheating or attempted cheating on their systems. Among those, Adair said:
The percent of exam questions found in online services and accessible to students has more than tripled in the past few years. In 2020, 12% of test questions were found in online searches, this year the rate is 36%. More than one in three online exam questions, in other words, is already compromised.
Honorlock said that 13% of students try to navigate away from a test screen during the test.
Three percent of students show up to tests with browser extensions enabled and “ready to use.” These extensions, Adair said, are commonly used for misconduct.
Three percent of test-takers also attempt to use “cut and paste” features during an exam.
Over 70% of “human-verified” incidents of exam misconduct, Adair says, “were tied to a second device” which is to say - cell phones or tablets or similar items.
Adair also said that they’ve found that shortening the exam time helps reduce cheating:
The longer in an exam, the likelihood of cheating increases
First, kudos to Honorlock for sharing what they know about exam conduct. It’s rare and it’s important.
Second, that more than one third of all online exam questions can already be found online is pretty shocking. Not surprising, but still shocking.
The 13% number is also an eye-opener because that represents students who tried to access something outside the exam setting, while they were using a lock-down browser. They knew someone was recording it, they knew their online activities were being monitored and prevented, yet still more than one in ten test takers tried to breach the test security parameters.
Sure, I can see some students checking to see if it’s working, that it works. But what if it didn’t? What would those inquisitive souls have done? Moreover, if it’s 13% inside Honorlock, can you imagine what it is without a lockdown browser?
Three percent cut-and-paste: again, that’s in a lock-down browser. And that’s not “testing the system.” That’s closer to outright attempted cheating.
I’m sure there’s some overlap in these percentages, but three percent plus three percent plus 13 percent is nearly 20% of students having a browser extension, trying to cut-and-paste, or trying to leave the exam platform. About one in every five. Again, for the third time - that’s while they’re using a lock-down browser.
Amazing.
Maybe I should have known that Honorlock had a feature Adair called “search and destroy” that checks for compromised exam questions online. But I did not. According to the presentation, the tool not only flags leaked questions, it generates automated take-down notices to get them removed. Pretty cool. I knew some other folks had been working on this, but I did not know Honorlock had it.
As for the AI portion of the presentation, it was underwhelming.
The solicitation for the webinar said they’d used ten different AI-checking systems and would share reports on how well they did. But they didn’t really do that. They took “an average” of the detection scores from detectors they did not name. So, not too helpful.
It was troubling too for Honorlock to demonstrate how to bypass AI similarity checkers. At one point, Adair said what he was doing was:
tips for the trade to get around detection tools
He said what he was showing was:
an easy way around
That’s not helpful.
I get why Honorlock wants to paint AI detection systems as “unreliable.” That’s a business decision. But every single integrity solution is hackable, able to be bypassed. Showing people exactly how to do it is not the best idea, even if it is good for business.
Still, I’m happy to have these data points.
Honorlock says you can access the on-demand recording here.
Course Hero’s Appalling Proctoring Panel
I’m sorry, this is long. But get your popcorn, I think it’s worth it.
Well, Course Hero has done it - something so shameless and inexcusable that all I can do is share it and say, without reservation, that it should disqualify the company from future civil discourse on the topics of education or integrity.
In the last Issue, I touched on Course Hero’s upcoming and incredible summit - incredible here means not credible.
If any educators or others were wavering on whether appearing on stage with cheating profiteer Course Hero was a good idea (see Issue 218), they should know who will also be on the same stage, under the same logo.
The Panel
Kind of buried on day two’s agenda is a panel titled:
Beyond Proctoring: Building Trust in an Online Learning Environment
First, pretty ballsy for Course Hero to use “trust” in anything.
This is the company that refuses to comply with academic integrity inquiries and, when asked to identify students who’d used Course Hero to cheat an exam, told the professor to get a court order. Which he did, by the way (see Issue 102). This from a company that CISCO designated as an “academic fraud provider” and blacklisted (see Issue 42). This from the company that gave me answers to questions I clearly said were for a test, during the test (see Issue 97).
Trust, sure.
It should be hot-air-balloon obvious why Course Hero has issues with online test security and proctoring - why they hate it. Pretty simple really, if an exam is supervised, cheating is harder. And they sell cheating. They’d much prefer no one watch students take tests.
It’s a trust thing. Sure it is. Like, I promise I am not a bank robber or anything, but wouldn’t it be better if banks trusted their customers and took down those terrible surveillance cameras?
But that’s not the appalling and disqualifying part. That’s coming.
Course Hero VP and Proctoring
On the panel is Sean Morris, the VP of Academics at Course Hero. Before joining Course Hero, Morris made a name for himself by opposing - wait for it - exam proctoring. In an interview in 2022, Morris said, before Course Hero:
My focus has primarily been on things like learning management systems and proctoring services and plagiarism-detection services and that sort of thing
Having issues with proctoring and plagiarism-detection makes it so odd that he’d end up working for a cheating provider. Genuinely puzzling.
In August of 2022, when a reporter pointed out to Morris that many colleges and universities block Course Hero from their systems because of cheating, he said (see Issue 143):
As a representative of Course Hero, I would say, try to get your university to not block us anymore. That would be great.
He later said that was a joke. Very funny.
But that’s not the appalling part either.
Shea Swauger
On the panel with Morris is Shea Swauger, who Course Hero identifies as:
a Senior Researcher for Data Sharing and Ethics at the Future of Privacy Forum. Previously, Shea was an academic librarian at the University of Colorado Denver.
Further, it says he:
has written numerous publications on privacy, ethics, education, and technology.
And that’s where the problems are.
What Mr. Swauger has written is not only blindingly incorrect, it’s offensive. Dangerous and offensive and appalling and disqualifying. If you can think of a few more bad adjectives, I’m sure they apply.
I hate giving him and his rants any attention whatsoever, but Course Hero put Swauger on an education panel to discuss proctoring and highlighted his writing. So, it feels fair to share what he’s written, what Course Hero brings to the table by brining him to the table. And I want there to be no question as to what he thinks or whether Course Hero is at least passively endorsing it by giving him a spotlight in which to repeat it.
His Blog Post
Swauger’s most infamous writing is this blog post from 2020, which I touched on briefly in Issue 166, because it had unbelievably made its way into bogus research. A problem for another day.
To be fair, the post is mostly about something he calls “algorithmic test proctoring,” which he incorrectly describes as:
essentially software designed to automatically detect cheating in online tests
But we can’t even escape the very first paragraph before Swauger writes:
If I take a test using an algorithmic test proctor, it encodes my body as either normal or suspicious and my behaviors as safe or threatening.
Encodes his body? Encodes his behaviors as threatening? I have no idea what he’s talking about.
And while I could discredit literally every line of his blog post, I don’t have the patience or stomach for it any more than I want to untangle fishing line with a three-year-old.
But Course Hero wants you to hear him. So, you should know what he’s written, what we may assume he believes. As such, I will touch on as few highlights as possible.
He wrote:
most proctoring software’s default settings label any bodies or behaviors that don’t conform to the able-bodied, neurotypical ideal as a threat to academic integrity.
Nope. Software does not label anyone as a “threat to academic integrity” let alone because of their body. Just does not happen.
He goes on:
Facial recognition literally encodes the invisibility of Black people and the racist stereotype that all Asian people look the same.
Ah, it literally encodes invisibility. Got it. And, what now?
As criticisms of proctoring go, so far, these are pretty banal and repetitive. Exaggerated, but regurgitated. And if he ended there, I would not write a single word. If someone wants to criticize how facial ID systems work and how some proctoring companies use them to verify the identity of a test-taker before a test, that’s fair.
But, We’re Just Getting Started
He also writes that proctoring software potentially misidentifying a student’s chosen name based on a student ID may:
[risk] more trauma and discrimination including being denied financial aid, being forced to leave their institution, or have their lives put in physical danger.
And … now we’re off the deep end. Someone please send me a postcard.
He continues that schools use proctoring because:
these companies are offering a product that resonates with several implicit values and practices of higher education that ultimately outweigh the risk to student safety: discriminatory exclusion, the pedagogy of punishment, technological solutionism, and the Eugenic Gaze.
And:
Higher education in the United States has feared including marginalized people from the beginning and test proctoring companies market directly to that fear. Their promotional messaging functions similarly to dog whistle politics which is commonly used in anti-immigration rhetoric. It’s also not a coincidence that these technologies are being used to exclude people not wanted by an institution; biometrics and facial recognition have been connected to anti-immigration policies
And:
test proctoring companies are communicating firstly, that non-traditional students, students of color, international students, and students typically excluded from higher education are threats because they are more likely to cheat and need to be held accountable
To paraphrase, Course Hero’s panelist thinks that an “implicit value and practice of higher education” is, among others, “discriminatory exclusion.” He says again that “higher education” has “feared including marginalized people” and that schools use test supervision technology “to exclude people not wanted.”
N.B., he’s not knocking test proctoring, he’s talking about higher education itself. In his world, teachers use test proctors to supervise exams so they can deny non-traditional students access to education - simply because he thinks that is what schools do.
And I really, really do wish I was done. But Swauger isn’t.
On Nazism, Forced Sterilization, Genocide and Test Proctoring
In fact, he’s not nearly done. In the same blog post, Swauger continues:
Eugenics programs attempt to remove people who have “undesirable” traits through anti-immigration policies, selective breeding programs, marriage restrictions, forced sterilization, murder, and genocide.
Higher education is deeply complicit in the eugenics movement. Nazism borrowed many of its ideas about racial purity from the American school of eugenics, and universities were instrumental in supporting eugenics research by publishing copious literature on it
Eugenics. Selective breeding. Sterilization. Genocide. Nazism.
And online test proctoring.
Got it. Makes perfect sense.
We see Swauger’s view again - that Universities and professors proctor their tests because they want to practice the kind of exclusion and racism that can be found in genocide and forced sterilization. In other words, a professor who teaches an online Intro to Chemistry class to 140 freshmen and wants to supervise their exams to prevent cheating is complicit in Nazi ideology and eugenics.
I think that’s what he’s saying. It’s hard to tell for sure, honestly. We are through the looking glass here. Either way, this cannot be real. But it is. And yet, he’s not still not done.
Swauger continues:
Algorithmic test proctoring uses the Eugenic Gaze by measuring student’s bodies and behavior (machine learning and facial recognition software), defining what bodies and behaviors are associated with the ideal student (cisgender, white, able-bodied, neurotypical, male, non-parent, non-caretaker, etc.), attempts to reform students who deviate from the ideal student (flagging them as suspicious), or exclude them from the community (academic misconduct investigations which can lead to expulsion). The Eugenic Gaze is a combination of white supremacy, sexism, ableism, cis/heteronormativity, and xenophobia. When we apply the Eugenic Gaze using technology, the way we do with algorithmic test proctoring, we’re able to codify and reinforce all of those oppressive systems
I see - “algorithmic test proctoring uses the Eugenic Gaze.” And that, Swauger says, is “a combination of white supremacy, sexism, ableism, cis/heteronormativity, and xenophobia.” Proctoring measures student bodies and attempts to “reform” students who are not male, white or non-parents.
Roger.
Swauger summarizes:
Algorithmic test proctoring is a collection of machine learning algorithms that reinforce oppressive social relationships and inflict a form of data violence upon students. It encodes a “normal” body as cisgender, white, able-bodied, neurotypical, and male. It surveils students and disciplines anyone who doesn’t conform to “normal” through a series of protocols and policies that participate in a pedagogy of punishment, ultimately risking students’ academic career and psychological, emotional, and physical safety.
Ladies and Gentlemen - your Course Hero panelist. Appearing with a corporate VP and sharing the stage with actual teachers from institutions such as Harvard and Columbia.
I just don’t know what to say.
Before we leave this single blog post by Swauger, I want to mention that in it he also describes “cheating, plagiarism, and citation” as “culturally constructed” and “seemingly arbitrary.”
It’s not just that test proctoring is evil and violent and oppressive and tied up with eugenics and Nazis and so on - it’s also that Swauger believes that cheating is flatly irrelevant. It’s his entire second theme and, I’d argue, the most important to understand.
Cheating Doesn’t Matter
On this theme, Mr. Swauger wrote yet another blog post which he titled - no I am not kidding:
Cheating Doesn’t Matter
The first line is:
I don’t care about cheating.
No, really. Go check.
Lest you think it’s some game of rhetoric he’s playing and he’ll later retreat, nope. He continues:
In school, it doesn’t matter the grade, the subject, or the test, I don’t think it’s a big deal.
And:
Every college and university academic integrity policy in the world, to my knowledge, disagrees with me on this. They regard cheating as a serious offense that can lead to the expulsion of the offending student. I’m undeterred.
He also writes:
Academic integrity brands itself as the pursuit of fairness (myth), rigor (dog whistle), and the assurance of merit-based (privileged) credentials (capitalist tokens).
To paraphrase the guy Course Hero invited to their event - fairness, rigor and merit-based credentials are kind of garbage. Got it.
He says of education itself:
Education is not, and never has been, a level playing field. It perpetuates the same social forces that govern most of our institutions: white supremacy, sexism, ableism, racial capitalism, transphobia, and jingoist propaganda.
Education, he says, perpetuates white supremacy, sexism, ableism and so on. So, cheating it is fine.
He says we should:
let go of academic integrity as a priority
And there it is.
Cheating does not matter because, Swauger seems to believe, education itself is corrupted. And since you can’t cheat a cheater, we should not care about it. Just amazing.
Those are just two of Mr. Swauger’s writings. There are more. They’re no better.
At the End
So, if you were wondering why Mr. Swauger is on this panel with Course Hero, it’s because he does not care about cheating and he hates exam supervision. Those are all the qualifications he needs to be Course Hero golden.
Course Hero doesn’t care that he compared proctoring to eugenics. Or thinks it is a tool of eugenics, more accurately. Or that he thinks education itself is sexist, ableist, racist and so on. Course Hero does not care that he invokes genocide and Nazism in a conversation about test proctoring.
They don’t care that he doesn’t care about cheating and thinks we should “let go” of academic integrity. Well, Course Hero probably actually loves that part.
They only care that he will bash academic integrity efforts and dismiss cheating. Pretty on brand if you ask me - just perfect.
I only hope Mr. Swauger doesn’t suddenly come upon a “scheduling conflict” at the last minute. Course Hero needs to own this. It’s who they are.
Update: Teacher Who Seeded Quizlet with Fake Answers
A few weeks ago, we shared the story of a professor who, after discovering his proprietary test materials on Quizlet, had them removed and planted obviously fake answers to his exams instead. As a result, 40 students were caught cheating (see Issue 214).
I reached out to the professor and asked him what happened to those 40 students.
Here is what Professor Merriam of California State University, Sacramento said by e-mail:
Of the 40 students who I suspected of cheating, 35 of them confessed. They all got zeros on the final, which resulted in many (but not all) of them failing the class. They will also be reported to our Student Affairs office. If they’ve been reported before, or if they ever are again, the University will sanction them in some way I know not how. If this is their first and only offense, however, then the grade is the only punishment they will receive.
Of the 5 that remain, one of them was right ‘on the bubble.’ I spoke with her over Zoom, and she convinced me that she was innocent. The remaining four that insist on their innocence have, by my calculations a 1:100,000,000 chance of being innocent. I’m handing over their cases to the University to investigate and deal with as they see fit.
Great update.
I hope the University takes these cases seriously and word of fake, seeded answers on cheating sites gets around.