Do you regularly visit your favorite weather Web site, or check the forecast every morning on your cell phone? Some of us really obsess about the weather. In the United States, we have an entire television channel dedicated to it, and I have friends who pay more attention to this channel than the news, their favorite sports team…perhaps even their spouse!
When you check the weather, you can just get the next-day forecast, or you can look ahead at a 5-day or even a 10-day forecast. But have you ever wondered how truly trustworthy these predictions are? Many people count on these long-range forecasts to accurately predict future weather, but are they really reliable, or are they more like the meteorological equivalent of gazing into a magic crystal ball?
In this article, we will look at temperature data and use various statistical tools to see just how reliable the forecast is.
We visited a leading weather Web site and collected the predicted next-day, 5-day and 10-day high temperatures (in ºF) for State College, Pennsylvania—the location of Minitab’s World Headquarters. We did this every day for 30 days, and recorded the forecasted temperatures, in addition to the actual high temperature for each day, in Minitab Statistical Software.
We recorded the 10-day, 5-day and next-day forecasts in Minitab, then calculated the difference between the forecasted and actual high temperatures.
The graph below shows that the forecast was more reliable on some days compared to others. It also shows the rises and falls in temperature that occurred in State College during the late spring.
We can use a Time Series Plot to compare the forecast to the actual temperature for each day of the study.
The individual value plot shows that the 10-day forecast exhibits more variation than the other two forecasts.
With a standard deviation of 6.2 degrees, we can see that the 10-day forecast overestimated the high temperature by as much as 8 degrees and underestimated it up to 17 degrees, as shown in the graph below. The 5-day and next day forecasts were less variable with standard deviations of 4.3 and 2.1, respectively.
Now, is this difference in variability between the 3 forecasts statistically significant, or are these observed samples likely given that the variances are truly equal? Let’s use an equal variances test to find out.
Conclusion: The next day forecast is significantly more precise than the other two forecasts.
Conclusion: The next- and 5-day average differences appear to be equally accurate, with both confidence intervals including 0.
The average disparity for the next-day forecast is significantly less than the 10-day forecast.
Now that we’ve done our comparisons and determined that the 10-day forecast is significantly less accurate, let’s evaluate just how well the 5-day and next-day forecasts can be used to predict whether you should plan to wear a swimsuit or a sweater.
We can use a fitted line plot to explore the relationship between the actual temperature and the 5-day forecast.
The p-value of 0.000, found in the Session window, indicates that a significant linear relationship exists between the actual temperature and the 5-day forecast.
The R-squared value tells us that this model accounts for 77% of the variability seen in the actual high temperature, which is likely better than your average crystal ball.
Also, using residual plots (not shown) we can verify the model assumptions and conclude that the analysis is valid.
Now, let’s run a similar analysis for the next-day forecast. The graph below and high R-squared value indicate that the next day forecast is a better and very reliable predictor of the actual high temperature.
Because the points fall close to the line, the regression model appears to be a good fit.
Using prediction intervals, we can calculate a likely range of values for a given next day forecast. For example, we can be 95% certain that a next-day forecast of 80ºF will likely correspond to an actual temperature between 75 and 83ºF.
It’s important to note that although regression tells us if a linear relationship exists, it does not tell us if this is a 1-to-1 relationship. In other words, our p-value could be significant because a forecast of 80ºF indicates an actual high of 80ºF, or it could be significant because a forecast of 80ºF does a good job at predicting an actual high of, say, 40ºF (e.g., if the slope coefficient is 0.5).
We can use a confidence interval to assess the coefficients for the y-intercept and slope.
Because the confidence interval for the constant (-1.75, 10.85) shown in the output above includes 0 and the confidence interval for the slope coefficient (0.85, 1.01) includes 1, we can conclude that the relationship between the next-day forecast and the actual temperature is in fact a 1-to-1 relationship.
Given all of the factors that influence it, the weather is an undeniably complex process—and like any process, it can exhibit a lot of variation. However, if you’re going to make any big plans based on weather and you want to minimize the variation, the data we collected suggest it’s best to rely on the next-day forecast.
There is not much we can accurately predict 5 days into the future, so relatively speaking, the 5-day forecast comes a lot closer to doing that than most aspects of life. As for the 10-day forecast, it’s likely that meteorologists know exactly how unpredictable the weather conditions 10 days in the future can be. And they provide it to us weather-watchers nonetheless because we still want some sense of what the future holds, despite the unreliability of the predictions. But it’s good to know which forecasts we can really count on, and which come closer to fortune-telling!
Michelle ParetProduct Marketing Manager, Minitab, LLC.
Eston MartzSenior Creative Services Specialist, Minitab, LLC.
Download this article as a PDF file
Get our free monthly e-newsletter for the latest Minitab news, tutorials, case studies, statistics tips and other helpful information.
Data is the new gold: 5 ways to make sure your data is reliable
Advancing the Power of Analytics
A Statistical Analysis of Boston’s 2015 Record Snowfall
By using this site you agree to the use of cookies for analytics and personalized content in accordance with our Policy.