New York City public schools have limited access to ChatGPT, the AI system that can generate text across a range of topics and styles across school networks and devices. As wide reported this morning and confirmed to businessroundups.org by a spokesperson for the New York City Department of Education, the restriction was implemented due to concerns about “[the] negative effects on student learning” and “the safety and accuracy” of the content that ChatGPT produces.
“While the tool may provide quick and easy answers to questions, it doesn’t build critical thinking and problem-solving skills, which are essential for academic and lifelong success,” the spokesperson told businessroundups.org via email, adding that the limited access came in response to requests from schools.
It’s not necessarily a ban. The New York City public school system uses the same filter for ChatGPT which it uses to block other apps and websites – YouTube and Facebook, for example – on school grounds. Individual schools can request that ChatGPT be unblocked, and the spokesperson said the New York City Department of Education would “welcome” the opportunity to have a conversation with OpenAI, the startup behind ChatGPT, about how to adapt the tool for the education.
As for OpenAI, a company spokesperson, when reached for comment, said OpenAI is developing “mitigations” to help anyone recognize text generated by ChatGPT. That is significant. While businessroundups.org recently reported that OpenAI was experimenting with a watermarking technique for AI-generated text, it’s the first time OpenAI has confirmed it’s working on tools specifically for identifying text coming from ChatGPT.
“We made ChatGPT available as a research preview to learn from real-world usage, which we believe is an essential part of developing and deploying capable, secure AI systems. We are constantly incorporating feedback and lessons learned,” said the OpenAI spokesperson. “We have always called for transparency around the use of AI-generated text. Our policy requires users to be aware of their audience when using our API and creative tools… We look forward to working with educators on useful solutions and other ways to help educators and students benefit from AI.”
ChatGPT has the aptitude for answering questions on topics ranging from poetry to coding, but one of its biggest shortcomings is its ability to sometimes provide answers that sound convincing but are actually untrue. That led Q&A coding site Stack Overflow to temporarily ban users of sharing content generated by the AI, saying that ChatGPT made it too easy for users to flood the platform with questionable answers. More recently, the International Conference on Machine Learning, one of the world’s largest AI and machine learning conferences, announced a ban on papers containing text generated by ChatGPT and other similar AI systems for fear of “unforeseen consequences”.
In education, the debate largely revolves around the potential for cheating. Do a Google search for “ChatGPT to write school papers” and you’ll find plenty of examples of educators, journalists, and students testing the waters using ChatGPT to complete homework assignments and standardized essay tests. Wall Street Journal columnist Joanna Stern used ChatGPT to write a passing AP English essay, while Forbes contributor Emma Whitford psycho it to finish two college essays in 20 minutes. Speak to The Guardian, Dan Gillmor, a professor at Arizona State University, recalls giving ChatGPT one of the assignments he usually gives his students and finding that the AI’s essay would have earned “a good grade.”
Plagiarism is another concern. Like other text-generating AI systems, ChatGPT – which is trained on public data, usually collected without permission – can sometimes spit out this information verbatim without citing any source. That includes factual inaccuracies, as mentioned earlier, as well as bias – inclusive shameless racist and sexist — perspectives. OpenAI continues to introduce filters and techniques to avoid problematic text generations, but new solutions are popping up every day.
Despite these limitations and issues, some educators see pedagogical potential in ChatGPT and other forms of generative AI technologies. In a recent piece for Stanford’s Graduate School of Education website, Victor Lee, an associate professor of education at Stanford, noted that ChatGPT can help students “think in ways they don’t currently,” for example, by helping them discover their ideas and clarify. Teachers can also benefit from ChatGPT, he argues, by generating many examples for students of a story in which the basic content remains the same, but the style, syntax or grammar differ.
“ChatGPT is allowed [allow] students to read, reflect, and revise many times without the anxiety or frustration that such processes often evoke, [while] teachers can use the tool as a way to generate lots of examples and not examples of one form or genre,” Lee said in a statement. “Teachers are of course less happy that the computer does a lot of work for students. And students still have to learn how to write. But in what way, and what kind of writing? A… side effect of this new drug is that we all have to ask those questions and probably make some substantial changes to the overarching goals and methods of our instruction.
In any case, the New York City public school policy, which appears to be the first of its kind in the country, is sure to force conversation in school districts elsewhere. As adoption of the technology grows – ChatGPT had over a million users in December – independent researchers and businesses have begun testing tools to detect the use of AI-generated text in student submissions. Some educators choose to embrace them, while others, like Lee, instead encourage the use of ChatGPT as a writing aid.
