AI model breaks the two-week weather forecast barrier

Breaking through the predictability desert

Weather forecasters have long faced a frustrating gap. Short-range forecasts of a few days are reliable, and long-range climate outlooks can track slow patterns like El Niño. But the period between two and six weeks is known as the "predictability desert" — forecasts there have been barely better than guessing ¹. A team of researchers has now developed a machine-learning technique that significantly improves predictions in that gap, doubling accuracy compared to existing methods.

The approach was published in Nature and placed first in a prominent international forecasting competition last year ¹. When the researchers applied their method to a gold-standard forecasting model, the improvements were dramatic enough to beat climatologic averages — essentially outperforming simply relying on historical norms for that time of year.

Learning from mistakes

The idea builds on earlier work in chaos theory by MIT Professor Edward Lorenz, who in the 1970s showed that tiny changes in initial weather conditions can lead to wildly different outcomes, creating a theoretical limit on how far ahead weather can be predicted ¹. Professor Judah Cohen, a co-author of the new paper based in MIT's Department of Civil and Environmental Engineering, pushed back against the idea that nothing could be done beyond two weeks .

"In theory the atmosphere can create an infinite number of patterns, but it likes to fall into these favorite patterns over and over again," Cohen says. "Once one thing happens, I think you can predict the domino that will fall after that, and the domino that will fall after that" ¹.

The team's solution was to build an AI layer on top of existing forecasting models. Instead of trying to make the models themselves better, they focused on quantifying and correcting the errors those models make. The system learns from historic forecast mistakes and adaptively adjusts new predictions as more data becomes available ¹. Other forecasting systems do simpler error correction, but this approach uses AI trained on decades of past data to recognize systematic patterns in the mistakes.

Why it matters now

Better two-to-six-week forecasts could be life-saving. In 2021, severe winter storms hit Texas and caused the worst energy infrastructure failure in state history, killing at least 246 people ¹. If officials had known those storms were coming weeks in advance, they could have moved more natural gas trucks to affected areas and potentially reduced the damage .

The improvements also extend to extreme events. "It's really important to predict both [average conditions and extremes], but extreme events might be the most impactful part of this for society," Cohen says ¹. Flooding, cold spells, and other severe weather patterns could be flagged earlier, giving farmers, energy companies, and emergency responders more time to prepare.

The model was trained on MIT Engaging, a supercomputing cluster, but the final system does not require a supercomputer to run ¹. The research involved scientists from MIT, Microsoft, the European Center for Medium-Range Weather Forecasts, DTN, Harvard, and the University of Toronto.

Broader context

Numerical weather prediction has relied on powerful supercomputers running complex math equations since the 1950s, but even today's best models only reliably predict about six days ahead ². Post-processing techniques have tried to clean up errors before, but this new AI-based method represents a substantial step forward for the subseasonal range — a sweet spot that sits awkwardly between daily weather forecasting and long-term climate prediction.

Key facts

• The AI model doubles forecasting accuracy for the two-to-six-week range
• It was trained on historic forecast errors and adapts as more data arrives
• Better subseasonal forecasts could have warned about the 2021 Texas winter storms weeks in advance
• The model also improves predictions of extreme weather events like flooding and cold spells
• The research team includes scientists from MIT, Microsoft, and the European Center for Medium-Range Weather Forecasts

Sources

1. Researchers improve weather forecasts looking two to six weeks out
2. Numerical weather prediction — en.wikipedia.org
3. Weather — en.wikipedia.org