Research: Honor Pledges Don't Work to Reduce Cheating Conduct

Plus, Course Hero's bizarre support. In two parts.

Issue 108

To subscribe to “The Cheat Sheet,” enter your e-mail address below:

To share “The Cheat Sheet:”

Thanks for sharing “The Cheat Sheet.” If you enjoy or support my work, please consider chipping in via Patreon:

New Research: 70% of Students Cheated on Unproctored, Online Exams. Honesty Pledges Ineffective.

New research out last month, ought to turn some heads about what works, and what does not work, to reduce cheating during online exams.

It’s by Jacob Pleasants of the University of Oklahoma and John M. Pleasants and Barbara P. Pleasants of Iowa State University.

Rather than repeat it, here’s this from the abstract:

We found that 70% of students were observed cheating, and most of those who cheated did so on the majority of test questions. Appealing to students’ honesty or requiring them to pledge their honesty were found to be ineffective at curbing cheating. However, when students received a warning that we had technology that could detect cheating, coupled with threats of harsh penalties, cheating behavior dropped to 15% of students.

The research team also found that student test scores did not change when their cheating behavior did. Though, they also found that it did.

In my view, there are some issues with their study. The sample size was just 66 students in a summer course, for example. And the authors’ definition of “cheating” isn’t proven cheating, it’s suspicious behavior detected by a learning management system - time away from browser, for example. The authors do acknowledge that and did take care to eliminate outliers in the data, making the 70% metric only the most likely to have cheated. Again, what they found is probably cheating, but it cannot be said to be conclusively.

I am also not keen on the absence of a real control group. The very first exam used “pledge” and “appeal” language on cheating. There was no test with no intervention.

But take my views here skeptically. I am not a trained researcher. Maybe my issues aren’t really issues.

What I do like is that the study used several different approaches to try to curtail misconduct. First, as mentioned, they used appeal language at the top of the tests:

It is important for the integrity of this course, the meaningfulness of grades, and fairness to other students that you do not use notes or any other materials while taking these tests.

In separate test groups, they asked students to actively affirm that they had not used outside materials in the test. They altered the time limits on the exams to test whether time pressures could influence the behavior that appeared to be - probably was - cheating.

None of that worked. The authors say:

Midway through the semester, after Exams 1 to 4 were completed, we found that none of our measures were effective at curbing cheating behaviors.

So the team tried a threat - a bluff really. They e-mailed students the following “deliberately vague” message and affixed it to the header of the final four exams:

This is a warning that due to concerns about students cheating on tests we now have the capability of monitoring student activity while taking tests. If I detect suspicious behavior on any of the remaining tests, I will have to take administrative action.

REPLY TO THIS E-MAIL TO LET ME KNOW YOU UNDERSTAND THIS WARNING

They also followed up individually with students whose learning management logs showed the most suspicious test behavior - telling them directly about suspicious behavior and threatening action if it continued.

After the broad, vague warnings, the suspicious behavior dropped from around 70% of test-takers to about 18%. After the more specific and individual follow ups, it dropped to about 11%.

Since the same tests were given in previous semesters, but in a test center with live proctors, the research could compare. Indeed, they found that scores in the online, unproctored tests - even with the pledges and appeals - the ones in which they found cheating behavior, were higher than those done with a proctor - .78 standard deviations higher. More evidence that cheating behavior is reduced by live proctoring.

And even though the research found that test scores did not decline as cheating behavior changed, they also kind of found that it did:

when a warning of surveillance was issued beginning with Exam 5, we found that the prevalence of cheating declined dramatically. If cheating were responsible for the elevated test performance seen in Exams 1 to 4, then the cessation of cheating should coincide with a decline in test performance. Indeed, we did find that average test scores declined along with the prevalence of cheating.

It was at the individual student level that this correlation dissipated.

There’s plenty more to say about this study. For example that, after the vague “monitoring” warnings and the drop in cheating behavior, some students stopped their behavior only for a short time - presumably because nothing happened. Quoting the authors:

this finding does raise the possibility that students might stop taking warnings of surveillance seriously over time

That feels important.

I’ll share two more quotes from the study and move on. One:

In the absence of warnings of surveillance, we found cheating behaviors to be widespread. Neither appealing to students’ academic integrity nor requiring an honesty pledge were found to be effective

And two:

Unfortunately, our results indicate that cheating is the norm rather than the exception.

More Bizarre Support for Course Hero, Part I

Cheating provider Course Hero is a difficult foe for professors, schools or others to battle. It’s worth billions and spends millions on some pretty sophisticated marketing (see Issue 106).

Part of that marketing effort is clearly aimed at trying to convince people it’s a legitimate education company. It’s not. I mean, no legitimate education company would refuse to identify its users and refuse to cooperate with inquiries into cheating (see Issue 107). But it sure would like you to believe it is.

That’s why it puts on shows of partnering with professors and others who will smile beneath the Course Hero banner (see Issue 99 or Issue 44). Their latest such effort is coming up this week, on April 13, and a new name has been added to the roster:

Suman Balasubramanian, Associate Professor of Mathematics, DePauw University

There are others standing by Course Hero, but I try to highlight the educators - people who should know better.

I mean I really don’t know what to do with a Course Hero speaker from “Rad Roller,” which, when you Google it, returns results for massage equipment.

After finding the right outfit, I guess it sells massage equipment. And maybe they have education classes too - not sure. But I do see that Rad Roller has an “affiliate program” in case you’re interested. And that “Dr. Grant’s Back Bundle” for $125 - that looks like a must-have.

If you do decide to spring for the Back Bundle or the Rad Roller “Neuro Ball,” tell me before you buy. They offer commissions.

My point is that Rad Roller and Course Hero are nice company for DePauw University and the University of Wisconsin Madison and others. And do be sure to tune in this week to hear what the dude for Rad Roller has to say about “Effective Assessment for Students and Teachers.”

More Bizarre Support for Course Hero, Part II

Cheating provider Course Hero is a difficult foe for professors, schools or others to battle. It’s worth billions and spends millions on some pretty sophisticated marketing (see Issue 106).

Part of that marketing effort is clearly aimed at trying to convince people it’s a legitimate education company. It’s not. I mean, no legitimate education company would refuse to identify its users and refuse to cooperate with inquiries into cheating (see Issue 107). But it sure would like you to believe it is.

EdTech Digest is an education outlet, though they may be best known for their heavily marketed annual “EdTech Awards.” The 2021/22 finalists and winners were announced a week or so ago and our good friends from Course Hero were well represented.

Course Hero was named an award finalist in the categories of “higher education solution,” “student study tools solution,” and “test prep solution.” Course Hero CEO Andrew Grauer was also named a finalist for “founder(s)/CEO - startup.”

And the winner of the 2022 “EdTech Award” for “student study tools solution” is:

Couse Hero

That’s right - the company that was named an “academic fraud” provider, (see Issue 42) just won an award as a “student study tool solution.” The company that won’t cooperate with academic integrity inquiries, that one.

Personally, I don’t think students are confused as to what Course Hero is and what it does. It sells answers to homework and test questions, even entire tests and pre-written essays. But if students are confused at all, it’s little wonder why.