NFL Predictive Ratings Update: Reduced Preseason Rating Weight

NFL Predictive Ratings Update: Reduced Preseason Rating Weight

December 5, 2016 – by David Hess

If you follow our NFL predictive ratings closely — and who doesn’t, really — you may have noticed that a few teams seemed intuitively off. The most obvious examples are Dallas and Arizona.

The Arizona Cardinals started the season #1 in our 2016 NFL preseason rankings, and despite a mediocre performance and a 5-6-1 record, had only fallen to #6.The Dallas Cowboys, on the other hand, began the season ranked #25 and had only risen to #7 after 12 games, despite a league-best 11-1 record.

Sure, we started the season believing Arizona was much better than Dallas. But how could Arizona still be ranked higher, after all we’ve seen this season?

This felt wrong to us. But we don’t make changes to our NFL predictive ratings based on feelings. We make them based on data.

So, we looked at the data.

What we found was that we needed to decrease the weight of our preseason ratings. Therefore, that’s what we’ve done. The change goes into effect today (December 5th, 2016).

Changing The Preseason Rating Weight

Until now, we’ve determined how to weight our preseason rating by looking at how accurately it forecasts a team’s end-of-season NFL predictive ratings. However, that’s somewhat analogous to only looking at the last game of the season in order to determine the weight.

We have a few years of history to look back on, now, when trying to determine the correct weight. That history allows us to determine a weight that improves predictions throughout the season, rather than just the end of the year. When taking this broader view, it appears that, yes, the preseason rating is being weighted too strongly.

The following chart shows 3 things:

The ideal preseason rate weight (relative to average performance in previous games) needed to optimize future predictions, in each game of the regular season (BLUE)The previous preseason rating weight used in our ratings up until now (GREEN)The new preseason rating weight we’re going to use going forward (RED)

You can see that this change significantly reduces the impact of the preseason ratings. (The red squares are a lot lower than the green triangles.) It also fits the observed data better. (The red squares are closer to the blue diamonds than the green triangles are.)

While this change does make the preseason ratings less important, their weight never decays to zero. Even at the end of the season, our NFL predictive ratings are more accurate when we take the preseason ratings into account than when we don’t.

One final note: you may notice that the chart above seems to show the observed ideal weight of the preseason rating dropping very low in Weeks 11 to 13, before rebounding later in the season. We plan to look closer at this in the offseason, to determine if we need to make additional changes.

If you liked this post, please share it. Thank you! Twitter Facebook