Researchers at the University of Reading in the U.K. have aired concerns about the integrity of tests after they surreptitiously submitted unedited artificial intelligence-generated exam answers, fooling exam markers and beating most of the human students.

In the study, the researchers created 33 fake student identities and used ChatGPT-4 to answer 63 questions in legitimate online assessments for undergraduate psychology courses containing short and essay-style answers. The professors on the course, who marked the exams, were not told about the experiment. The AI-generated students, on average, did better than the human students. The fact they were AI “verged on being undetectable,” said the researchers.

Only one of the papers handed in was flagged as possibly not being human. The authors of the study, who called it “the largest and most robust blind study of its kind” regarding trying to pull the wool over educators’ eyes, said this surely must mean the AI passed the “Turing Test” – when humans aren’t able to distinguish a computer from a human.

Dr. Peter Scarfe, one of the study’s authors and an associate professor at Reading’s school of psychology and clinical language sciences, called the study “particularly worrying,” adding that it should stand as a “wakeup call” to educators who at some point are very likely going to get fooled by their students whose exam responses are the work of regular generative AI models.

“Many institutions have moved away from traditional exams to make assessment more inclusive,” explained Scarfe. “Our research shows it is of international importance to understand how AI will affect the integrity of educational assessments. We won’t necessarily go back fully to handwritten exams – but the global education sector will need to evolve in the face of AI.”

Concerns were somewhat ameliorated by the fact the AI was not as good at answering questions on tests given to third-year students, which required more abstract reasoning. However, for tests given to first- and second-year students, ChatGPT romped home. Since the results of the study were published, experts have said this could spell the end of take-home course work.

Image: Freepik