Foundation Models
Recalculating the climate with AI
Picture: UFZ
Helmholtz researchers want to develop an AI foundation model that enables more long-term and reliable climate forecasts. The project is part of an initiative that aims to leverage the potential of these models for research.
The climate is changing. Globally. Looking back into the archives is important in order to identify key trends, however, we need to look into the future in order to best prepare society, nature and infrastructure for future climate scenarios and to stabilize the climate. Although modern technologies and complex models have improved weather and, above all, climate forecasts in recent decades, the predictions are still prone to error. Martin Schultz and his team want to change this, with the help of artificial intelligence (AI).
Schultz is Professor of Computational Earth System Science at the University of Cologne and a scientist at the Jülich Supercomupting Center (Forschungszentrum Jülich). He also heads the research initiative HClimRep, one of four pilot projects of the Helmholtz Foundation Model Initiative (HFMI). Over the next three years, Schultz and his colleagues will develop a foundation model that is able to deliver better climate forecasts than conventional models. For the first time, information about the atmosphere, oceans and sea ice will be combined in an AI model. "To be able to predict the climate properly, you have to take into account the interactions between these components," says Schultz.
Foundation models are special machine learning models that are trained with extremely large data sets. Through comprehensive but still relatively unspecific basic training, they learn to independently recognize complex patterns and correlations. In the next step, they can apply this extensive knowledge to a variety of different tasks, so-called downstream tasks, for which they were not originally trained. For these tasks, they often only require a small amount of additional, specific data in order to function effectively. This distinguishes Foundation Models from conventional AI applications.
AI-based models benefit from many billions of data points from decades of measurements and analyses of temperature, winds and clouds. "The Foundation Model uses all this data to independently recognize correlations between individual parameters, and it does so better than a human ever could," says Schultz. "You could almost say that an AI model like this develops a physical understanding of the climate, even though it has never seen the equations that meteorologists use to describe the climate system." As early as 2021, Schultz, together with Christian Lessig from the European Center for Medium-Range Weather Forecasts and Ilaria Luise from CERN, began developing the first foundation model for the highly complex dynamics in the atmosphere: AtmoRep. The current HClimRep project builds on this.
[Translate to Englisch:] Supercomputer am Forschungszentrum Jülich. Komplexe KI-Anwendungen wie Foundation Models sind ohne die Rechenpower dieser Rechner nicht möglich. Bild: Forschungszentrum Jülich
AI models have shaken up meteorologists in recent years because they can make more accurate weather forecasts much faster than classic models. These AI weather models have been trained for a specific task: To produce weather forecasts. "Our new deep learning model will work differently," explains Schultz. The initial basic training will take place without a specific task for the foundation model. Only in the next step will it be trained to predict long-term changes in trace substances in the stratosphere, simulate marine heatwaves in different climate zones and improve regional flood forecasts, for example.
One of the biggest challenges for the team is that HClimRep must be able to process a wide variety of data formats and extend the forecasting period from days to spans of several decades. "It is not yet possible to predict how far into the future we will be able to look with HClimRep," says Schultz. "The latest studies suggest that this may work. But even if we can only make seasonal predictions in the end, that's still a huge step forward." One potential difficulty in achieving even longer predictions are climatic tipping points. "These can lead to weather events that we have never experienced before. As a result, the Foundation model cannot get to know them during training and is unlikely to be able to draw any sensible conclusions from the data," says Schultz.
All in all, a huge task for the next three years. The Alfred Wegener Institute, Helmholtz Center for Polar and Marine Research, the Karlsruhe Institute of Technology, the Helmholtz Center Hereon and the Jülich Research Center are all involved. At the Jülich Research Center which hosts Europe's first exascale computer, the HClimRep model is being trained. Even if the training of such a model requires a lot of computing power, it is more efficient overall and therefore saves resources than current climate models. This is because they require a lot of computing power to simulate different climate scenarios, whereas the pre-trained HClimRep model will be extremely fast and efficient.
The expectations are high. "However, we won't be able to develop a completely new climate model in the next three years," says Schultz. But there will already be decisive progress and impressive results. The entire research community will benefit from these. After all, all HFMI models and data will be made available as open source in accordance with the FAIR principle.
Helmholtz Foundation Model Initiative
Foundation models are a new generation of AI models that have a broad knowledge base and are therefore able to solve a range of complex problems. They are significantly more powerful and flexible than conventional AI models and therefore hold enormous potential for modern, data-driven science. They can become powerful tools that answer a wide range of research questions. The Helmholtz Association is ideally placed to develop such pioneering applications: a wealth of data, powerful supercomputers on which the models can be trained, and in-depth expertise in the field of artificial intelligence. Our goal is to develop foundation models across a broad spectrum of research fields that contribute to solving the big questions of our time.
Readers comments