OCW: Many instructors have adopted AI policies either along the lines of “No generative AI ever, at any point!” or else “Feel free to use generative AI, as long as you disclose/cite the nature of its contribution to your work.” Your policy, by contrast, suggests that students either “don’t use it” or else “use it a lot.” How has this rather quirky policy worked in practice?
Justin Reich: I don’t know. This is an important point. Knowing if a policy works requires a substantial amount of investigation, and I haven’t had the time (in terms of sufficient length of time for the policy to play out) or the time (in terms of free hours to investigate) to discover more.
In CMS.100, I decided to try to ban it for the first couple of assignments. My core theory is this: MIT students almost universally write good sentences and paragraphs, and they really need help with argumentation and structure. Basically, they are still writing five paragraph essays—sometimes literally five paragraphs.
Getting beyond these trivial structures requires students to develop their theses and arguments, and for most students the only way to advance these ideas is to write about them. So, many essays will have this wonderful quality where the introduction proposes a rather bland argument, but somewhere towards the end of the essay, you can find a much better version of the argument of the paper. This isn’t surprising, they thought through the issue and were smarter after writing four pages than they were when they started. But if they don’t do that intellectual work, I can’t help them see how their refined thinking at the end of the paper should actually be in the introduction and the basis for improving the structure of the whole paper. Asking students to do that work independently is vital to my approach to coaching them towards improvement.
I’m very concerned that co-writing with GPTs will smother this developmental process—that the GPTs will do too much thinking, and the reasoning will be too even throughout the paper, it won’t develop. So I banned it.
I didn’t find any evidence that my students used GPTs anyway, but I think it’s quite possible that they did, and I didn’t notice it or catch it.
In CMS.595 this year, I more or less made my students do the “use it a lot” version. I would generally describe the experiment as unsuccessful, but not terribly so. The first assignment is to have students pick an edtech product, decide whether it aligns better with Instructionism or Social Constructionism (the two main learning theories), and then imagine how it would be redesigned if the other camp had designed it. (For instance, if they picked Scratch, they should recognize its social constructivist origins, explain them, and then imagine what Scratch would look like if designed by Instructionists.)
You can’t Google that assignment, but ChatGPT can do a decent job. So this year, I required students to make ChatGPT write drafts for six to ten edtech products, and then pick one draft to refine. The theory was that the GPT-powered assignment lets them consider the comparison across more cases than they could do writing from scratch alone.
Overall, the assignments were not better than past years. Students were not supercharged learners accelerated by AI. They had about as many errors and mistakes as I might expect, which is really good. When students have errors, I can help them! However, in past years, when students wrote correctly and produced thoughtful analyses, I could trust they understood the underlying material. In this version, however, I’m not sure. Maybe they just copied it from GPT, and they got lucky and copied something right, but they don’t really get it.
I did have four students who submitted separate assignments that each used an identical example, sometimes with identical language. They all suggested as an activity that students do a “market day” as a kind of project-based learning. In reading maybe 150 of these papers in the past, I’d never seen that example. For reasons that are slightly complicated, it’s an OK but not great example / piece of evidence for a paper. So GPT seemed to have found a local minima in its statistical model of language, and nudged my students in.
I’m suspicious that the learning that students did during this process was not as good as if they’d written the whole thing on their own, but it’s hard to know. More research is needed, as the saying goes!
