Court Refuses to Set Aside Discipline of Student for Submitting AI-Generated Assignment (with Hallucinations)

From Harris v. Adams, decided Wednesday by Magistrate Judge Paul G. Levenson (D. Mass.):

In December 2023, school officials at Hingham High School (“HHS”) determined that RNH and another student, both of whom were juniors at the time, had cheated on an AP U.S. History project by attempting to pass off, as their own work, material that they had taken from a generative artificial intelligence (“AI”) application. Although students were permitted to use AI to brainstorm topics and identify sources, in this instance the students had indiscriminately copied and pasted text from the AI application, including citations to nonexistent books (i.e., AI hallucinations).

The students received failing grades on two parts of the multi-part project, but they were permitted to start from scratch, each working separately, to complete and submit the final project. By way of discipline, RNH was required to attend a Saturday detention, and in the spring of 2024, he was rejected from the school’s National Honor Society, although he was ultimately permitted to reapply and has since been admitted. {RNH, a current senior at HHS, is described in the Complaint as a “three-sport varsity student-athlete with a high [GPA]” who is “in the top of his class.” In addition to having a high GPA, RNH received a 1520 on the SAT and a perfect score on the ACT, putting him “in the top 1/4 of 1% of students taking the [ACT].” Plaintiffs report that RNH intends to apply early decision or early action to elite colleges and universities, such as Stanford University.}

Plaintiffs, RNH’s parents, have sued HHS teachers and school officials, also naming the Hingham school committee as a defendant. Invoking the Due Process Clause …, they ask this Court to undo the consequences that school officials imposed. Plaintiffs contend that HHS failed adequately to inform RNH about its standards for academic honesty as they apply to the use of AI, that Defendants were unfair in concluding that RNH had violated the school’s academic integrity policies, and that HHS imposed unduly harsh consequences. Plaintiffs ask the Court to expunge RNH’s disciplinary record (i.e., the Saturday detention) and to raise his grade in AP U.S. History from a C-plus to a B.

Defendants, in response, point out that RNH was repeatedly taught the fundamentals of academic integrity, including how to use and cite AI. Defendants argue that HHS officials reasonably concluded that this case did not implicate subtle questions of acceptable practices in deploying a new technology, but rather was a straightforward case of academic dishonesty. Defendants emphasize that, in any event, the Constitution does not empower judges to substitute their judgments for those of teachers and school officials, who are afforded broad discretion when it comes to grading and discipline.

Plaintiffs have moved for a preliminary injunction. Given that RNH is a senior, and that many colleges and universities have already begun accepting early decision or early action applications, Plaintiffs argue that RNH will suffer irreparable harm if the relief he seeks is not granted on an expedited basis (i.e., before he needs to submit college applications)….

The court denied the preliminary injunction, concluding that plaintiffs “Defendants have the better of the argument on both the facts and the law.” The opinion is over 16,000 words long, but here’s a key excerpt:

On the facts, there is nothing in the preliminary factual record to suggest that HHS officials were hasty in concluding that RNH had cheated. Nor were the consequences Defendants imposed so heavy-handed as to exceed Defendants’ considerable discretion in such matters.

As detailed below, school officials could reasonably conclude that RNH’s use of AI was in violation of the school’s academic integrity rules and that any student in RNH’s position would have understood as much. The work in question was a script for a short documentary film, which RNH and his partner submitted for an AP U.S. History project assigned in conjunction with the National History Day organization. The evidence reflects that the pair did not simply use AI to help formulate research topics or identify sources to review. Instead, it seems they indiscriminately copied and pasted text that had been generated by Grammarly.com (“Grammarly”), a publicly available AI tool, into their draft script.

Evidently, the pair did not even review the “sources” that Grammarly provided before lifting them. The very first footnote in the submission consists of a citation to a nonexistent book: “Lee, Robert. Hoop Dreams: A Century of Basketball. Los Angeles: Courtside Publications, 2018.” Docket No. 23-4, at 1. The third footnote also appears wholly factitious: “Doe, Jane. Muslim Pioneers: The Spiritual Journey of American Icons. Chicago: Windy City Publishers, 2017.” Significantly, even though the script contained citations to various sources—some of which were real—there was no citation to Grammarly, and no acknowledgement that AI of any kind had been used.

On the law, it is doubtful that the Court has any role in second-guessing the judgments of teachers and school officials who are responsible for grading and disciplinary decisions, particularly discipline short of suspension. There is no dispute that RNH, as well as his parents, were afforded prompt notice of the school’s findings and were given an opportunity to be heard. This is the kind of process that the Supreme Court has deemed sufficient for more substantial punishments than what RNH received. Moreover, Plaintiffs have not shown any misconduct by school authorities, let alone misconduct so egregious as to satisfy the applicable “shocks the conscience” standard….

Defendants could reasonably consider that RNH had been taught that all sources—including AI sources—must, at a minimum, be cited. See Docket No. 24-8 (instructing students to “give credit to AI tools whenever used, even if only to generate ideas or edit a small section of student work”); Docket No. 24-9, at 16 (instructing that AI “must be cited” if used by students). In these circumstances, Defendants could also have inferred that, if RNH had sincerely believed that he was permitted to use AI tools like Grammarly to generate text and include that text as his own, he would have cited the AI tool he used.

The manner in which RNH used Grammarly—wholesale copying and pasting of language directly into the draft script that he submitted—powerfully supports Defendants’ conclusion that RNH knew that he was using AI in an impermissible fashion. The purpose of the Assignment, plainly, was to give students practice in researching and writing, as well as to provide students an opportunity to demonstrate, and the teacher an opportunity to assess, the students’ skills.

Considering the training provided to HHS students regarding the importance of citing sources generally, Defendants could conclude that RNH understood that it is dishonest to claim credit for work that is not your own. Although, as discussed below, the emergence of generative AI may present some nuanced challenges for educators, the issue here is not particularly nuanced, as there is no discernible pedagogical purpose in prompting Grammarly (or any other AI tool) to generate a script, regurgitating the output without citation, and claiming it as one’s own work. See Docket No. 24-8 (noting that AI tools should not be used to “replac[e] [the students'] own critical thinking”) (emphasis in original).

At the preliminary injunction hearing, RNH testified that he was “confused” about the rules regarding use of AI, both generally and on the Assignment. He testified, for example, that he did not understand at the time that the instruction he received in ELA class with respect to use and citation of AI applied to his other classes, such as AP U.S. History. In his testimony, RNH also suggested, albeit somewhat equivocally, that an additional source of confusion was that he had been unable to access the National History Day rules through the link provided in the Assignment.

There is, however, nothing in the record to suggest that RNH told his teacher that he was confused or that he had been unable to use the link for the National History Day rules. Defendants could reasonably infer that a high school student who was genuinely confused about the rules governing an assignment would be capable of asking his teacher for clarification, particularly when the student had been unequivocally instructed (albeit in a different class) that “[i]f there is a question about when, where, and how to use [AI] tools, the student must communicate with their instructor in advance of use.” See Docket No. 24-8 (emphasis in original).

In light of the evidence developed to date, RNH’s testimony that he was “confused” smacks of after-the-fact rationalization. As noted above, in June 2024—six months after the Assignment, but prior to the commencement of this lawsuit—RNH described his understanding as follows:

“When she (Ms. Petrie) assigned the project, AI was not specified. In our English class, they talk about it. If you’re going to use it, say why and how. Use your own intuition about right and wrong. If you are to use it, you need to identify that you’re using it. It was an academic honesty point. Be honest and transparent with how you’re using it.”

These words reflect that RNH was capable of understanding, and did understand, that his training on the use and citation of AI was not simply a technical requirement for a particular class. In his own words, “[i]t was an academic honesty point.” Some matters covered in ELA class were undoubtedly specific to that class. For instance, RNH expressed uncertainty during his testimony as to whether the Assignment required citations in conformity with the Chicago Manual of Style, as opposed to MLA format. But he plainly understood that fundamental principles of “right and wrong”—which would not vary from class to class—were involved as well.

In sum, there is nothing to suggest that Defendants unreasonably jumped to conclusions when they determined that RNH had attempted to pass off AI-generated language as his own and, in so doing, had violated the school’s standards for academic integrity. “Defendants’ decision that some discipline was warranted,” therefore, “cannot fairly be characterized as an arbitrary exercise of authority.” As for the discipline itself, it is not the role of this Court to second-guess the determinations of teachers and administrators about the academic and disciplinary consequences that should be imposed for violations of a school’s academic integrity policies….

Plaintiffs contend that RNH’s conduct did not constitute plagiarism because text generated by AI is not attributable to any particular human author. They contend, in other words, that AI is not an “author” whose work can be stolen; it simply “generates and synthesizes new information.” Plaintiffs buttress this argument by emphasizing that various industries—including “academia and … the legal profession”—are “still grappling with how to address [AI's] use” and that “there is much dispute as to whether the use of generative AI constitutes plagiarism.”

Despite Plaintiffs’ strenuous efforts to frame this case as one of “first impression in the Commonwealth” about how to deal with an emerging technology, the Court need not parse the terms of the Handbook as if it were a criminal statute to decide whether Grammarly can reasonably be considered an “author” as the term is used in the Handbook. The Supreme Court has expressly eschewed such an approach…. “Given the school’s need to be able to impose disciplinary sanctions for a wide range of unanticipated conduct disruptive of the educational process, the school disciplinary rules need not be as detailed as a criminal code which imposes criminal sanctions.” …

In any event, the Handbook defines plagiarism as “the unauthorized use or close imitation of the language and thoughts of another author and the representation of them as one’s own work.” Even if I were to credit RNH’s testimony that he was “confused” about what uses of AI were permitted, it strains credulity to suppose that RNH actually believed that copying and pasting, without attribution, text that had been generated by Grammarly was consistent with any standard of academic honesty.

Since long before the advent of AI, and even before the advent of the printing press, there have been plenty of works whose origins are sufficiently obscure as to raise serious doubts about whether they can be considered the work of any “author” at all, or whether they simply reflect a syntheses of multiple strands of text and information that have been merged, by processes only partially knowable, into individual “works.” The Bible, Beowolf, and the works of “Homer” come to mind. The Handbook definition of plagiarism seems adequate to alert students that they may not copy such works without attribution and pass them off as their own.

If more were needed, it could be mentioned that Defendants found that RNH’s AI use violated the Handbook in three separate respects: (1) he used it in an effort to gain an unfair advantage over other students who did not use AI; (2) “he cheated by using unauthorized technology”; and (3) he committed plagiarism by “the unauthorized use or close imitation of the language and thoughts of another author and the[n] represented them as his own work.” Even if there were any legs to Plaintiffs’ argument that AI should not be considered an “author” whose “language and thoughts” may not be copied without proper attribution, there remain two additional violations of the Handbook.

Gareth W. Notis (Morrison Mahoney LLP) represents defendants.

The post Court Refuses to Set Aside Discipline of Student for Submitting AI-Generated Assignment (with Hallucinations) appeared first on Reason.com.