Executive Summary
The Japan Meteorological Agency (JMA) is responsible for forecasting weather conditions across the country. It is especially concerned about predicting dangerous events that threaten property and life, such as typhoons and torrential rains. Torrential rain events caused by linear precipitation zones have increased more than twofold over the last 45 years due to climate change.1
JMA uses weather prediction models the agency has developed internally running on several supercomputers for numerical weather prediction (NWP). The agency upgrades computing capabilities according to a technology development roadmap to increasingly enhance predictions to help protect property and save lives.
Following its technology roadmap, in 2023 JMA added a new supercomputer built by Fujitsu designed to improve prediction of torrential rain events caused by linear precipitation zones. In 2024, the agency added their 11th Gen supercomputer system from Fujitsu to enhance prediction of both linear precipitation zones and typhoons. This 11th Gen cluster, designed around the Intel® Xeon® CPU Max 9480 processor with High Bandwidth Memory, is twice as powerful as its predecessor, according to testing by the JMA.2
The new system has enabled JMA forecasters to improve predictions of both linear precipitation zones and typhoons and develop new products for its customers who need early information and warning about ongoing weather and dangerous weather events. The Intel Xeon CPU Max 9480-based system has also allowed them to reduce the number of nodes needed for some NWP workloads by as much as 80 percent.
Challenge
To continue advancements in weather prediction, JMA follows a plan to continually upgrade their technology over the next decade. The agency’s goals are to continue to provide more accurate predictions at higher resolutions to help with disaster prevention and safety measures, especially when it comes to catastrophic events, including typhoons and torrential rains caused by linear precipitation zones.
JMA has developed their own numerical weather prediction (NWP) models over the years, and the agency continues to enhance their forecasting for torrential rain events using data from new and varied sources across a network of sensor systems around the country.3 These new systems add to the already massive acquisition of data needed for weather modeling and prediction.
Designed around the Intel® Xeon® CPU Max 9480 processor with High Bandwidth Memory, JMA’s 11th Gen Supercomputer System is twice as powerful as its predecessor, enabling higher resolution, extended-time forecasts for typhoons and other catastrophic weather events.
While NWP has matured over the decades, linear precipitation zones are not yet a well-understood phenomenon by weather experts; numerical models to simulate them are still in the investigative and development stages. But they cause major disasters in Japan nearly every year and elsewhere around the world where conditions lead to building of well-developed cumulonimbus clouds organized in a linear pattern. According to a recent study, in the last 45 years, torrential rains have more than doubled in Japan, resulting in massive flooding, property damage, and loss of life.1
In March of 2023, JMA installed a new supercomputer built by Fujitsu that would allow them to continue to advance the accuracy of their weather forecasts, including linear precipitation zone predictability. That system was built on Fujitsu’s A64FX processor with 32 GB of high-bandwidth memory (HBM) per CPU. But, to further enhance their technology roadmap, they needed additional computing capacity to run other data-demanding numerical weather prediction workloads.
In March of 2024, JMA added yet another system built by Fujitsu and designed around the Intel Xeon CPU Max 9480 processor with 64 GB of HBM per socket—doubling the HBM capacity per socket over the previous system.
Solution
JMA’s weather models include a global model with a horizontal resolution of about 13 km, a meso model that covers Japan with a horizontal resolution of 5 km, and a local model with a horizontal resolution of 2 km. The global model is used for predicting the paths of typhoons, while the meso and local models are used for predicting heavy rains and linear precipitation zones across Japan.
Enhancing Weather Predictions with Intel Xeon CPU Max Series with HBM
“The majority of calculations used in the weather prediction models are limited by memory bandwidth,” stated Masafumi Chida, of the Database Department at the JMA. “New supercomputers with greater memory bandwidth provide incredibly fast memory usage by comparison to previous machines, allowing for much better efficiency.”
That efficiency translates into faster processing of large amounts of data and quicker results that can help accelerate preparation for dangerous weather events and evacuations.
To enhance the agency’s computation capacity, in 2024 the agency deployed a large computing cluster designed to handle its increasingly expanding datasets for NWP. The new Intel Xeon CPU Max Series processor-based system—referred to as the 11th Gen supercomputer system— integrates 64 GB of HBM2e on the processor. The system’s architecture includes the following:
- 496 Fujitsu PRIMERGY CX2550M7 nodes, including 12 reserve nodes
- Two 56-core Intel Xeon CPU Max Series 9480 processors per node with 64 GB HBM2e per processor
- InfiniBand 200 Gbps NDR in a Fat Tree configuration
- Lustre filesystem
“For a typhoon’s path, the estimation error by our numerical forecast model 72 hours in advance is around 200km, which is at the top-level standard compared to meteorological institutions worldwide,” said Chida. “Accurate estimation of a typhoon requires handling a massive set of data and immense precision in all processes pertaining to weather. A supercomputer capable of performing these operations is indispensable. Introducing a new supercomputer with this elevated performance will enable these functions, further reducing the error.”
JMA’s models are employed based on the scope and period of the prediction required. The global model sets the boundaries of the meso-model, and the meso-model defines the boundaries of the local model. Because these models are codependent, they must be employed as a unified package, with the appropriate schedule regimen to accurately model weather patterns.
Weather Prediction That Never Stops
“To always provide accurate weather forecasting, the new cluster is set up as two redundant systems, with a main system and sub-system,” explained Chida. “Typically, the main system is used for work, while the sub-system is used backup when the main system is down and for development at other times.”
According to Chida, a comprehensive evaluation with benchmark testing during a competitive procurement process led JMA to the new Fujitsu system. The Intel Xeon CPU Max 9480 processor was chosen due to the sheer memory bandwidth requirement needed for traditional weather models. According to JMA testing, the new system is about 2x more powerful than its predecessor.2
Intel Xeon CPU Max 9480 Processor
The Intel Xeon CPU Max 9480 processor is designed to unlock performance and accelerate discoveries in dataintensive workloads, including weather modeling, artificial intelligence (AI), and deep learning.
With 64 GB of HBM and up to 56 cores per socket per CPU (providing over 1 GB per core of HBM), Intel Xeon CPU Max Series processors deliver as much as 4.8x better performance compared to competitive devices on real-world workloads.4
Although JMA has not chosen to include them, the Intel Xeon CPU Max Series also supports up to 4TB of DDR5 memory in addition to HBM. Intel Xeon Max Series CPUs offer flexibility to run in different memory modes, or configurations, depending on the workload characteristics:
- HBM-Only Mode—Enabling workloads that fit in 64 GB of capacity and ability to scale at 1-2 GB of memory per core, HBM-Only mode supports system boots with no code changes and no DDR.
- HBM Flat Mode—Providing flexibility for applications that require large memory capacity, HBM Flat mode provides a flat memory region with HBM and DRAM and can be applied on workloads requiring more than 2 GB of memory per core. Code changes may be needed.
- HBM Cache Mode—Designed to improve performance for workloads needing more than 64 GB capacity or requiring more than 2 GB of memory per core. No code changes are needed. HBM caches DDR.
- High-bandwidth memory technology continues to advance with next-generation Multiplexer Combined Rank (MCR) memory (MRDIMM). These new modules multiplex two ranks of memory per DIMM, essentially doubling the speed of memory access. Internal Intel analysis reveals that MRDIMMs deliver up to 2-3× better performance for HPC workloads in comparison to current 5th Gen Intel Xeon processors with DDR5 DIMMs. Intel Xeon 6 processors are designed to support MRDIMMs.
oneAPI-ready
JMA’s deployment of the new system includes the Intel® oneAPI Base and HPC toolkits. The new weather prediction supercomputer can take advantage of all the benefits that Intel® oneAPI toolkits offer, with optimized libraries, compilers, and other tools that enhance computing on Intel® architecture. oneAPI enables a common, open, standardsbased programming model that unleashes productivity and performance across different Intel devices, including Intel CPUs and GPUs.
Result
Weather prediction with finer resolutions in the models helps reduce errors thereby improving forecasts. With additional observational data and enhanced computing capabilities that allow more and faster processing, JMA is able to deliver better forecasts by extending the specifications of existing numerical models. Additionally, the number of nodes required for some NWP workloads is fewer, freeing more computing resources for other workloads and projects.
“We extended the forecast time for our local model from 10 hours to 18 hours,” stated Chida. “We intend to increase this model’s horizontal resolution from 2 km to 1 km and begin new operation of a local ensemble forecast system. The high performance of the new supercomputer will contribute to the realization of this plan.”
Scientists can also create new weather prediction products and explore development of other advanced methods, including Artificial Intelligence (AI). Additionally, the new system with 64 GB of HBM per CPU allows JMA more flexibility to run different types of workloads (regional, global, deterministic and ensembles, coupled and standalone models, etc.) with HBM that would not fit in the confines of the previous A64FX-based supercomputer. With the ability to configure HBM on the Intel Xeon Max CPU in cache mode, applications with large memory footprint requirements, such as coupled atmosphere and ocean high-resolution global models, benefit from a very large and fast CPU cache—without developers needing to modify code to take advantage of HBM.
With the aid of improved disaster prevention information JMA expects its weather predictions to perform at a high level of precision and speed, starting with the critical prediction of linear precipitation zones.
“This new system is a major contributor to increasing precision of linear precipitation zone prediction,” added Chida. “As of 2023, our probability to predict linear precipitation zones to occur 15 hours in advance is at 33 percent. With the heightened resolution of the local model and new introduction of the local ensemble forecast system, more accurate prediction will be possible.”
Solution Summary
Japan Meteorological Agency’s critical mission helps protect life and property with accurate predictions of weather events using supercomputers and their weather modeling systems. NWP is already highly data intensive, yet new observational data was added to help improve accuracy of critical weather events, including linear precipitation zones. A new supercomputer, designed around the Intel Xeon CPU Max 9480 processor with HBM was deployed in March of 2024, designed for the high memory bandwidth demands of NWP. The new system allows JMA to deliver higher local forecast resolution, improve prediction of linear precipitation zones, and deliver new products to their weather information consumers.
