What ChatGPT means for education and science

No, because the call for a moratorium has generated a lot of public attention. It makes sense to have a broad debate now about how we want to deal with AI systems. Society needs to be clear about what it is getting into and what rules we want to agree on. The open letter was met with many reactions: Numerous experts pointed out the opportunities that such programs offer, while others emphasized that we sometimes overestimate the capabilities of programs like ChatGPT that sometimes appear more intelligent and thus more threatening than they actually are.

However, some cities, states and universities have even banned ChatGPT or only allow it in a restricted way. Do you think that makes sense?

In my view, a general ban is too sweeping, because systems like ChatGPT harbor dangers and potential in equal measure. On the one hand, they can take over tedious routine tasks for us, for example, or simplify access to knowledge. On the other hand, they can also provide fake news or reproduce certain prejudices, for example, by reinforcing gender stereotypes. Even for us as experts on the impact of technology, it isstill too early to make a final assessment of the impact of the programs, which we also point out in our recently published study.

Pressing questions are already being asked, for example, in research. The software is likely to be tempting for scientists who want to cheat.

Of course, this is a big issue! Programs like ChatGPT could increase the number of fraud cases in science, because there is already a lot of pressure to publish as much as possible. So it's easy to imagine researchers being tempted to let an AI system write their studies. However, many scientific publishers categorically reject this; after all, ChatGPT cannot assume any responsibility for the content of a text. At the same time, however, there are indications that the system could also be helpful in scientific writing, for example, when it comes to getting an overview of the relevant literature or publishing in a language other than one's native tongue.

Can ChatGPT already write entire studies or scientific articles?

In the future, such systems could not only formulate research results, but also design and conduct chemical experiments, as one study has shown. In this study, ChatGPT planned a series of experiments on a given question and forwarded them to an automated pipetting system. Although this was only intended to demonstrate feasibility so far, this study illustrates how fundamentally scientific practice could be affected. In contrast, the articles published so far, which were partly or largely written using ChatGPT or related systems, tend to show the limitations of the system as the results have not turned out to be all that original. But of course this can only be judged for the cases where the use of ChatGPT was made transparent.

Will it be technically possible to detect whether texts originate from an AI in the foreseeable future?

I highly doubt it, because ChatGPT mostly generates unique texts: The texts are not composed of set pieces from different sources, but they are, so to speak, reassembled word by word on each request. Existing software for detecting plagiarism thus fails here, and new programs are being trained, but so far they are running with only moderate success. More promising would be a kind of watermark: This involves interspersing certain patterns in the texts that do not bother us humans when we read them, but are recognizable by machines. However, there are still a number of technical challenges, and the developers of the AI systems must also play along and use the method in their systems.

How can we still assess performance fairly, for example, at schools and universities?

For example, by designing exams differently: Pupils and students no longer just hand in a finished text, which is then assessed as good or bad, but they exchange ideas with their teachers at a much earlier stage. For example, they discuss the development of a question, the search for sources, or the structure of an argument. An AI system like ChatGPT can only help to a very limited extent in dealing with sources. Even these intermediate steps are then assessed by the professors. The focus is therefore more on supporting the learners, which is particularly important in schools.

Why?

Because otherwise the programs could reinforce social inequalities. This has already been shown with other digital teaching and learning offerings: According to studies, it is primarily high-achieving students who benefit from such apps. Weaker children and young people tend to learn less effectively with them. Teachers also tell us similar things about ChatGPT. At TAB, however, we are also concerned about another problem: Data protection.

What risks do you see?

With ChatGTP, students are using a system that is managed by a private company in the USA. They may feed it with a large amount of very personal data and disclose information about their performance. We see that as very questionable. The Italian data protection authority has therefore even banned ChatGPT and is demanding improvements from the developers. However, we at TAB also see great opportunities in such programs.

Which ones?

Let's stay with schools for a moment: There, language-based systems like ChatGPT could help to prepare learning offers in a differentiated way. They then develop tasks for the same content, but at different levels of difficulty depending on the learning level of the respective child. I see a lot of potential here!

And at universities?

There, the programs are primarily of interest for subjects that work a lot with language, i.e., in the social sciences. Some teachers are already using ChatGPT like a sparring partner: The app drafts coherent counter-arguments to a position, which the students then have to respond to. - This trains their own thinking. The situation is more difficult in the natural sciences: Logical thinking and mathematical skills are central there, and ChatGPT has been weak in these fields so far.

Are you considering using the program in research as well?

Yes, for example, in medicine: AI could be helpful in genome analysis, because the models read genetic code in a similar way to human languages and recognize patterns and irregularities in it. However, this branch of research already uses other AI systems that are far more advanced. Whether ChatGPT outperforms their results remains to be seen. The system also designs program code - this capability could also be interesting for science, as researchers often use specially designed scientific software as well as smaller utilities. ChatGPT masters various programming languages and delivers code that, while not always directly usable, runs quite convincingly after a rework. Or it shows potential for improvement towards more elegant solutions. This is where systems like ChatGPT show their great value, because they make our work easier, not only in science, but also in many other areas.

Do you have an example?

Yes, legal advice. Lawyers often work in a very routine way, for example, when they check contracts. In the future, an AI system may be able to do this just as well; a pilot project is underway at the Stuttgart Higher Regional Court. Positive effects could also be seen, for example, in the inclusion of people with disabilities: There are still only a few texts translated into accessible language, and ChatGPT or related systems could make an important contribution here in the future. The system thus holds numerous opportunities, as well as risks, and we point out both in our study. It provides a juxtaposition of pros and cons, which cannot be avoided at the moment. At present, it is too early to make any firm decisions.