fields/climate-weather/model families
Transformers in weather, climate and oceans
How transformers, a kind of neural network that weighs which parts of a sequence matter to which, are used to forecast weather, seasonal climate shifts and ocean conditions.
How it is used
Weather, climate and ocean science works with fields of numbers spread over the globe and stepping forward in time: air pressure, wind, temperature at many depths or heights, rainfall. A transformer is a network built around attention, a step that lets every part of the input look at every other part and decide how much weight to give it. Applied here, the globe is cut into patches, each patch becomes a vector of numbers, and attention learns which distant regions and which earlier months bear on a given place. The network is trained on long archives of past observations and reanalyses, the gridded reconstructions of past weather, and then asked to roll the state forward.
Two uses dominate. One is short and medium-range forecasting: take the current atmospheric state and predict it some hours or days ahead, then feed that prediction back in to reach further. The other is seasonal and ocean prediction, where the target is an index of sea surface temperature or wind stress months in advance. The same machinery also serves nowcasting of rain from radar and the gap-filling of patchy station or flux measurements, often alongside convolutional layers that handle local spatial structure before attention handles the long-range links.
Why researchers use it
The usual tool is a physical model: equations of fluid motion and radiation solved on a grid. These are trustworthy and interpretable, but costly to run, and they need many runs to express uncertainty. A trained network stands in for that simulation at the forecast step, trading a long physics computation for a pass through a fixed set of learned weights. Once trained, it is cheap enough to run many times over.
Attention is attractive in this field because the connections that matter are not local. A wind anomaly in one ocean basin can bear on temperatures in another months later, and rainfall depends on conditions far upwind. Convolutions see mainly nearby cells; attention can link far-apart patches and far-apart times directly, which is why it is reached for when the signal is a teleconnection rather than a local pattern.
In the record
Limits
These models learn from the archive, so they reproduce its biases and its gaps, and rare extremes are thinly represented. Rolling a forecast forward by feeding predictions back in tends to blur fields and drift over long horizons. Nothing enforces conservation of mass or energy, so output can be physically inconsistent. Attention over many patches is memory-hungry, which pushes resolution down. And a model fitted to the past climate has no guarantee of holding in a shifted one, where the statistics it learned no longer apply.
Written by AIxSci: the background from general knowledge, the examples from checked records.
Related
Every article
aix-00283
Deep learning forecast of El Niño suggests Indian and Atlantic oceans extend warning time
aix-00292
Neural network forecasts daily sea surface temperature up to 90 days ahead
aix-00295
Diffusion model trained on past weather produces 15-day global ensemble forecasts
aix-00296
Neural networks trained on 39 years of weather data produce week-ahead global forecasts
aix-00298
Generative AI model forecasts UK rainfall 90 minutes ahead from radar images
aix-00299
Neural network forecasts weather straight from raw satellite and station observations
aix-00280