Do you regularly visit your favorite weather Web site, or check the forecast
every morning on your cell phone? Some of us really obsess about the weather. In
the United States, we have an entire television channel dedicated to it, and I
have friends who pay more attention to this channel than the news, their
favorite sports team…perhaps even their spouse!
When you check the weather, you can just get the next-day forecast, or you
can look ahead at a 5-day or even a 10-day forecast. But have you ever wondered
how truly trustworthy these predictions are? Many people count on these
long-range forecasts to accurately predict future weather, but are they really
reliable, or are they more like the meteorological equivalent of gazing into a
magic crystal ball?
In this article, we will look at temperature data and use various statistical
tools to see just how reliable the forecast is.
We visited a leading weather Web site and collected the predicted next-day,
5-day and 10-day high temperatures (in ºF) for State College, Pennsylvania—the
location of Minitab’s World Headquarters. We did this every day for 30 days, and
recorded the forecasted temperatures, in addition to the actual high temperature
for each day, in Minitab Statistical Software.
We recorded the 10-day, 5-day and next-day forecasts in
Minitab, then calculated the difference between the forecasted and actual high
The graph below shows that the forecast was more reliable on some days
compared to others. It also shows the rises and falls in temperature that
occurred in State College during the late spring.
We can use a Time Series Plot to compare the forecast to the
actual temperature for each day of the study.
The individual value plot shows that the 10-day forecast
exhibits more variation than the other two forecasts.
With a standard deviation of 6.2 degrees, we can see that the 10-day forecast
overestimated the high temperature by as much as 8 degrees and underestimated it
up to 17 degrees, as shown in the graph below. The 5-day and next day forecasts
were less variable with standard deviations of 4.3 and 2.1, respectively.
Now, is this difference in variability between the 3 forecasts statistically
significant, or are these observed samples likely given that the variances are
truly equal? Let’s use an equal variances test to find out.
Conclusion: The next day forecast is significantly more precise than the
other two forecasts.
Conclusion: The next- and 5-day average differences appear to be equally
accurate, with both confidence intervals including 0.
The average disparity for the next-day forecast is
significantly less than the 10-day forecast.
Now that we’ve done our comparisons and determined that the 10-day forecast
is significantly less accurate, let’s evaluate just how well the 5-day and
next-day forecasts can be used to predict whether you should plan to wear a
swimsuit or a sweater.
We can use a fitted line plot to explore the relationship
between the actual temperature and the 5-day forecast.
The p-value of 0.000, found in the Session window, indicates that a
significant linear relationship exists between the actual temperature and the
The R-squared value tells us that this model accounts for 77% of the
variability seen in the actual high temperature, which is likely better than
your average crystal ball.
Also, using residual plots (not shown) we can verify the model assumptions
and conclude that the analysis is valid.
Now, let’s run a similar analysis for the next-day forecast. The graph below
and high R-squared value indicate that the next day forecast is a better and
very reliable predictor of the actual high temperature.
Because the points fall close to the line, the regression
model appears to be a good fit.
Using prediction intervals, we can calculate a likely range of values for a
given next day forecast. For example, we can be 95% certain that a next-day
forecast of 80ºF will likely correspond to an actual temperature between 75 and
It’s important to note that although regression tells us if a linear
relationship exists, it does not tell us if this is a 1-to-1 relationship. In
other words, our p-value could be significant because a forecast of 80ºF
indicates an actual high of 80ºF, or it could be significant because a forecast
of 80ºF does a good job at predicting an actual high of, say, 40ºF (e.g., if the
slope coefficient is 0.5).
We can use a confidence interval to assess the coefficients
for the y-intercept and slope.
Because the confidence interval for the constant (-1.75, 10.85) shown in the
output above includes 0 and the confidence interval for the slope coefficient
(0.85, 1.01) includes 1, we can conclude that the relationship between the
next-day forecast and the actual temperature is in fact a 1-to-1
Given all of the factors that influence it, the weather is an undeniably
complex process—and like any process, it can exhibit a lot of variation.
However, if you’re going to make any big plans based on weather and you want to
minimize the variation, the data we collected suggest it’s best to rely on the
There is not much we can accurately predict 5 days into the future, so
relatively speaking, the 5-day forecast comes a lot closer to doing that than
most aspects of life. As for the 10-day forecast, it’s likely that
meteorologists know exactly how unpredictable the weather conditions 10 days in
the future can be. And they provide it to us weather-watchers nonetheless
because we still want some sense of what the future holds, despite the
unreliability of the predictions. But it’s good to know which forecasts we can
really count on, and which come closer to fortune-telling!
Michelle ParetProduct Marketing Manager, Minitab Inc.
Eston MartzSenior Creative Services Specialist, Minitab Inc.
Download this article as a PDF file
Get our free monthly e-newsletter for the latest Minitab news, tutorials, case studies, statistics tips and other helpful information.
Data is the new gold: 5 ways to make sure your data is reliable
Advancing the Power of Analytics
A Statistical Analysis of Boston’s 2015 Record Snowfall