The fastest code in the field of laser-plasma physics

There are few things that can really make the 18,688 graphic cards of the American supercomputer “Titan” break out in a sweat – but when the junior physicists at the Helmholtz-Zentrum Dresden-Rossendorf  (HZDR) show up with their code, even the second-fastest computer in the world has to rise to the occasion in top form. With a record performance of 7.1 quadrillion calculations per second (PFLOPS), physical 3D models arise that show what processes are taking place during particle accelerators. These findings should help to make devices for proton-radiation therapy against cancer more compact and more inexpensive.

Such complicated 3D simulations are necessary, because extremely complex processes in very short timeframes are being examined: When a laser, one-hundred-trillion watts strong, impacts with a foil at the HZDR Institute of Radiation Physics, a mixture of trillions of electrons and ions arises – high-energy plasma. This procedure takes place in fractions of a femtosecond, i.e. about the length of time it takes an electron to travel around the atomic nucleus. In order to represent the interactions between the particles in plasma in a model, a lot of computer power and an efficient programme code are required. The basis for this is provided by the code PIConGPU  developed at HZDR: “We can even track an individual particle under 100 billion electrons and calculate its influence upon the overall system,” explains Dr Michael Bussmann, Director of the junior-scientist group for computer-assisted radiation physics.

The code is based on the so-called “particle-in-cell” method (PIC): this involves the solving of differential equations that describe the interaction of electromagnetic fields with charged particles such as electrons and ions in a virtual-spatial grid. The plasma physicists are thus able to find out how the particles move in the cells of the 3D grid and retroact on the fields. But the Dresden scientists do not allow the PIC procedure to be executed by normal central processing units (CPUs); rather, they use the computing units of graphic cards – so-called GPUs. This GPU computing offers a critical advantage, as Michael Bussmann explains: “Modern graphic cards can carry out many calculation steps at the same time, and are hence able to calculate the processes in the particle chaos of the plasma faster than central processing units.” Since the supercomputer Titan and the HZDR data-processing centre both rely on GPUs, the Dresden code could be quickly adapted for simulations on the American mainframe computer.

The key feature here is the code’s high level of scalability. This means that the code’s computing power is proportional to the number of deployed graphic cards. This should not be taken for granted, as Heiko Burau, doctoral candidate in Bussmann’s junior-scientist group emphasises: “Even the shortest unused waiting times in data processing lead to a majority of the GPU performance being wasted.” For maximum performance, the graphic processors have to work independently of one another, if possible. Since thousands of cards are performing calculations together on the models, a challenge thus emerged: How does one compensate for slow data transfer between the individual graphic cards? The simple, but effective solution: During the transmission to other cards, new calculations are beginning right away. The waiting period is thus used wisely, and all of the graphic cards work at the same time.

Heiko Burau is also the inventor of PIConGPU: After receiving an award in the pupil competition “Youth conduct research”, he was given the opportunity to do research on his topic “graphic-card programming” at the HZDR. In only took six weeks and he had already programmed, with the other young research scientists in the group, the initial version of the code for an individual GPU. The junior-scientist group finally developed, step by step, the most powerful code in laser and plasma physics today.

Michael Bussmann also praises the flexibility of the code: “Whether it is cosmic jets in astrophysics, the examination of laser-driven fusion, or laser particle acceleration – in the meantime our code can execute almost any calculations in which high-energy plasmas play a role.” This is not least because it is an open-source code that can be used by every scientist free of charge and adapted to individual research. The HZDR research scientists are convinced that mutual exchange of knowledge will advance the development of PIConGPU even further, which is why they make the cheeky request: “Please steal this code!”

You can access all archived editions of HELMHOLTZ extreme here: www.helmholtz.de/extreme