Within approximately 30 minutes, organizations with an NVIDIA RTX PRO or data center GPU can begin using new tools within NVIDIA Earth2Studio to refine weather forecasting. The platform allows for the efficient processing of increasingly localized observations from sources like wind and solar assets, radar, and satellites, helping manage physical risk across sectors from energy to agriculture. These tools focus on two advanced techniques: constraining diffusion models with point observations and assimilating disparate datasets into a consistent state.
This capability enables users to update forecasts more rapidly than traditional numerical analyses, which require substantial processing time and fixed schedules. You can use Score-Based Data Assimilation (SDA) to incorporate observations into diffusion-based AI downscaling and forecasting models such as CorrDiff and StormCast. SDA guides the model toward predictions that are consistent with your observations without requiring retraining the model. Diffusion models generate high-resolution predictions through a sequence of denoising steps.
At each step, SDA compares the intermediate prediction with your observations and nudges the model in the right direction. The output of SDA is probabilistic, with less uncertainty near observation locations and a wider spread further away, where predictions are increasingly governed by the other model inputs and the underlying AI simulations. SDA uses the multi-step denoising process of diffusion models. The final output is an observation-informed, high-resolution weather prediction.
NVIDIA Earth-2 Enables Timely Weather Decision-Making
Energy companies can now refine regional forecasts by integrating observations from wind and solar assets using NVIDIA Earth-2, a platform designed to process localized data efficiently. The system facilitates incorporating proprietary or third-party data, enabling more frequent forecast issuance and alignment with real-time conditions, tailoring predictions to specific regions and applications, NVIDIA says. This capability is particularly valuable for managing energy production and demand across transmission corridors or densely populated areas. Earth2Studio provides a pretrained CorrDiff model over Europe that transforms 0.25° weather fields into 2.2-km predictions, a technique for AI-based downscaling.
A basic knowledge of Python and a development environment with Earth2Studio installed are also prerequisites for users seeking to use the AI data assimilation tools. To illustrate the platform’s capabilities, developers can explore end-to-end AI data assimilation examples within the Earth2Studio example library or follow the custom data source example to connect their own observations, according to NVIDIA. The system’s ability to incorporate observations relevant to a specific region or organization allows for more accurate and timely forecasts.
Score-Based Data Assimilation (SDA) Improves Regional Forecasts
Earth2Studio delivers a functional global data assimilation model pre-trained for research, integrating data from diverse sources onto a 1° HEALPix grid, HPX64, to provide a starting point for customized forecasting. This system ingests observations from microwave sounders, radio occultation, surface stations, aircraft, and buoys, offering a broad base for regional adaptation.
The platform’s flexibility extends to training regional systems instead of relying solely on global models, allowing users to focus computational resources where local accuracy is paramount. The speed of data assimilation is significantly improved through the use of HealDA, an AI-based technique capable of estimating global atmospheric state.
This rapid processing contrasts with traditional numerical methods, which often limit forecast timeliness and the ability to incorporate custom observations.
CorrDiff Downscaling Enhanced with Point Observations in Earth2Studio
Earth2Studio’s CorrDiff technique uses AI-based downscaling, transforming 0.25° weather fields into 2.2-km resolution predictions over Europe, and recent advancements demonstrate a pathway to refine these forecasts by incorporating real-time observational data. This approach allows for a guided diffusion process, steering predictions toward observed realities rather than relying solely on modeled projections. To quantify the impact of SDA, developers ran CorrDiff with and without observational constraints, measuring the resulting changes in forecast accuracy; the results revealed that in this example, SDA reduces the wind-speed root mean squared error, or RMSE, at held-out stations by 54%.
Earth2Studio’s StormCast model, pretrained over the contiguous U.S. and initialized with HRRR data, produces 3-km resolution predictions, and SDA allows for updating these forecasts with the latest observations, compensating for the time lag inherent in traditional HRRR analysis dissemination. Comparing 3- and 6-hour forecasts with and without SDA reveals a clear improvement in prediction accuracy when observational data is incorporated, further demonstrating the effectiveness of the technique.
Beyond pre-trained models, Earth2Studio facilitates the assimilation of observations with custom models by extending the platform’s model wrapper, allowing developers to tailor the system to their specific needs and data sources. The team performs two runs to measure how the additional observations affect the results, calculating a “prior” forecast from free downscaling and an “analysis” forecast guided toward the observations, as demonstrated in example code available within the Earth2Studio library.
SDA Integrates Proprietary Data for Focused Weather Predictions
Earth2Studio expands forecasting capabilities by allowing incorporation of proprietary, regional, or domain-specific observations into existing numerical analyses; this moves beyond reliance on broad-based datasets alone. The system’s effectiveness, however, is contingent on observation quality, spatial distribution, and the scale of the predicted field, alongside a well-defined observation operator. This flexibility enables organizations to refine forecasts for specific asset locations or downstream applications, tailoring predictions to their unique needs, the company says.
CorrDiff, an AI-based downscaling technique available within Earth2Studio, transforms 0.25° weather fields into 2.2-km resolution predictions, and its integration with SDA creates high-resolution initial conditions for even more detailed forecasting via StormCast. A custom PyTorch model allows assimilation of observed impacts beyond standard meteorological data; for instance, a wind power model can integrate turbine output measurements directly into the forecast. The example implementation, based on the StormCast CONUS wrapper, demonstrates this capability.
Earth2Studio’s data assimilation system utilizes an observation encoder that treats each data stream as a point cloud, transforming measurements into tokens processed by a ViT backbone. Users can use a pretrained global data assimilation model as a starting point, and typically incorporate custom conventional observations without model modification. Adapting the encoder to support proprietary satellite data further expands the system’s versatility, allowing for tailored data assimilation specific to a region or application, NVIDIA reports.
This approach extends beyond regional systems, enabling the training of global systems with customized data streams. The open-source Python library PhysicsNeMo provides the HealDA training pipeline to facilitate this process. Detailed examples and a user guide are available to help users explore end-to-end AI data assimilation and the broader capabilities of AI weather models within the Earth2Studio environment.
StormCast-SDA Updates Forecasts Beyond HRRR Analysis Schedules
Earth2Studio’s StormCast-SDA system allows for forecast updates at one-hour intervals, a cadence enabled by combining the latest observations with existing analyses, a marked departure from the fixed schedules governing traditional numerical analyses. This approach bridges the temporal gap between analysis release and current conditions, offering a more responsive forecasting capability, particularly valuable when relying on a global analysis with a six-hour dissemination schedule.
The system’s core functionality centers on an operator that interpolates grid values to observation locations, but extends to accommodate proxy measurements; for instance, wind turbine power output can be a proxy for wind speed itself. at a 3-km resolution, makes predictions initialized with HRRR data, but Earth2Studio users can limit the domain to specific regions, such as the central U.S., by defining latitude and longitude boundaries.
Model configuration allows for adjustments to the number of diffusion steps, with 96 steps allocated for SDA to enhance stability, while the initial model loading process itself is streamlined for efficiency. Fetching HRRR initial conditions and defining observation data sources, like GHCN-Hourly stations, are key steps in preparing the system for operation. The system’s workflow involves an initial rollout phase incorporating observations, followed by a forecasting period proceeding independently, mirroring a pipeline where observations bridge the gap between analysis cycles, the company’s account states.
A single SDA-informed step is often sufficient when initialized with HRRR, but multiple rollout steps can benefit from SDA when using a global analysis for initialization. The process begins by initializing a generator with HRRR data and then iteratively sending observations to advance the forecast, as demonstrated by the code sequence x, coords = gen.send(obs).
The Earth2Studio example library provides a complete implementation of this workflow, allowing users to explore the system’s capabilities firsthand. This approach isn’t merely theoretical; the system demonstrably improves wind speed predictions, as evidenced by comparisons between SDA-enhanced forecasts and those generated without SDA. The ability to continuously incorporate observations, rather than waiting for scheduled analyses, represents an advancement in the speed and responsiveness of weather forecasting, offering organizations a pathway to more timely and accurate predictions.
See today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals.
