One Class Used AI, One Didn’t. Their Exam Scores Were the Same

"It's not that AI helped students learn more-it helped them learn more efficiently and confidently," says Rojas, who taught both sections of the course. "They spent less time outside the classroom on homework and exam preparation."

Standardized course evaluations completed at the end of the semester were also significantly higher in the AI section, particularly in ratings of instructor preparation and use of class time. In addition, students with AI access were far more likely to say they intended to pursue careers that involve intensive use of AI.

Rojas says the results should reassure educators that integrating AI into coursework can be accomplished without sacrificing academic rigor. He suggests a "permit with scaffolding" approach, where students are provided explicit instructions about effective uses of AI and clear disclosure requirements.

"There's an opportunity for instructors to be more open about AI usage. Letting students engage with it just creates a different environment," Rojas notes. "It's been super impactful for me and the way I think about teaching."

The experiment intentionally assigned the permission to use AI to an afternoon section, which historically performs slightly worse, making it a conservative test against detecting outsized AI effects. Students in both sections were assessed with paper-and-pencil exams, where notes, AI or other technology were not permitted.

Even though the study spanned a considerable amount of time, Rojas cautions that many of the outcomes were self-reported by students and that the experiment involved a small sample size.

The study is detailed in a new working paper, "Allowing Generative AI in Class: Evidence from a Semester-Long Controlled Teaching Study," which has been submitted for publication in a peer-reviewed journal.