Sunday, September 11, 2005

Cloudy forecasts

If it's the brink of the NFL season, you're sure to find newspapers, sports magazines and sports-oriented websites offering their predictions for the coming season. All have their picks for division winners, wild cards, league champions and the Super Bowl winner. It's all fodder for debate, and there's nothing wrong with any of it. (But doesn't anyone expect Pittsburgh to be better than Cincinnati this year?)

Some sources go further, however, and try to predict the regular season record of each team. Even the people who make these predictions know they're on a fool's errand. Indianapolis can't win 13 games if Peyton Manning breaks his leg. If Neil Rackers suddenly gets the yips, Arizona finishes with three fewer wins. Some team expected to win 10 games will in fact win five, and a team expected to win maybe five or six games will instead win 11 or 12 and get at least one playoff game at home. The best a prognosticator can hope to do is say Team A should be good, Team B will probably stink and Team C should improve. But that kind of equivocation doesn't sell newspapers, move magazine subscriptions or drive Web traffic. So you get exact forecasts of won-loss records.

Writers have their reasons -- good reasons -- for predicting that, say, Arizona will go 9-7 and win the NFC West. Or that Atlanta will go 13-3. Anything's possible, and I have yet to see a preseason prediction that defies all logic. No one is saying Cleveland will go 14-2 this year. Or predicting a 3-13 finish for New England. So it's pointless to quibble with specific predictions before a down has been played.

All that said, however, there are some things every NFL seer should be taking into account when dishing up his or her predictions for the year about to kick off. Basic stuff that should be central to any attempt to forecast exact records. Whether a season preview gets those things right -- now that's something worth critiquing.

I looked at six sources that predicted final records for the 32 teams: a national magazine, a national newspaper, two local newspapers and two independent websites. They are:
  • Sports Illustrated. Picks
  • USA Today. AFC | NFC (projected records appeared in print edition; not available online)
  • Tampa Tribune. Picks
  • Buffalo News. AFC | NFC
  • The Sports E-Cyclopedia. Picks
  • WhatIfSports.com Insider. Picks
The first issue that must be considered is league-wide record. Get a calculator and go to Doug Drinen's definitive pro-football-reference.com. Pick an NFL season and add up the wins of all the teams in the league. Then add up all the teams' losses. Compare the numbers. Try another year. Or let me save you the trouble: The numbers are always equal. Every game has a winner and a loser. With 32 teams, there are now 256 games in an NFL regular season. Thus, all forecasts of regular season records should total 256 wins and 256 losses. But hey lookee here:

FORECASTTOTAL WINSTOTAL LOSSES
Sports Illustrated256256
USA Today282230
Tampa Tribune256256
Buffalo News264248
Sports E-Cyclopedia260252
WhatIfSports.com255257

The results point to the dangers of a) looking at each team, division or conference in isolation, and b) splitting your forecast among several writers. If I'm responsible for breaking down only the AFC East, I'll say New England's going 13-3, the Jets will be 11-5, Buffalo 9-7 and Miami 5-11. Reasonable, right? But that's a division record of 38-26. Somewhere, some team or combination of teams has to lose 12 more games than they win in order to balance it out. Now, if several writers have divvied up the eight divisions, no one is looking at the big picture. And because the tendency is to give teams the benefit of the doubt, the balance sheet becomes skewed in favor of wins. USA Today had several writers assemble its forecast and wound up with a forecast 26 games out of balance. The Buffalo News had one writer for each conference and came out eight games out of whack. The Tampa Tribune, however, had one writer, Ira Kaufman, take on the whole league, and his forecast is balanced. Sports Illustrated also was in balance. Adam Hoff at WhatIfSports.com, a site that pits teams from different areas against each other, came close enough -- one game out of balance -- that it may just be a mistake. (Credit to Hoff for being the only one to err on the side of more losses, though.)

Another element a forecast should consider is the share of winners and losers. Though the number of wins and losses is always equal, the numbers of teams with winning records and losing records are rarely the same. Say you have a four-team league. You could have teams finishing 14-2, 7-9, 6-10 and 5-11. The league would have an even 32 wins and 32 losses, but would have just one winning team and three losing teams. Complicating matters is the fact that the NFL usually has some teams that finish as neither losers nor winners. These are the ones that go 8-8.

For a sense of what an "average" NFL season produces in terms of winners and losers, I looked at the past 15 years, 1990-2004. Raw numbers can't be compared from year to year because for the period in question, the league grew from 28 teams (1999-94) to 30 teams (1995-98) to 31 teams (1999-2001) to 32 teams (2002-present). So for each year, I converted the numbers of winners, losers and 8-8 teams to a percentage of the teams in the league. I then got the average of those percentages over the 15-year span and applied it to the current 32-team league.

On average, over the past 15 years, 45.3% of the teams in a given year finished 9-7 or better; 43.9% finished 7-9 or worse; and 10.9% finished 8-8. Projected to an "average" year in a 32-team league, that comes out to 14.5 winners, 14 losers and 3.5 teams at 8-8. Here's how the six forecasts compare with the hypothetical average year:

AVG.SIUSATTampaBuffaloE-CycloWhatIf
Winners14.5151717171516
Losers1412812121114
8-83.5573362

The only number here that's completely out of the range seen in the past 15 years is USA Today's multi-writer forecast of eight losers, or only 25% of the teams in the league. But here's what's really remarkable about the table:

Every forecast calls for more winners than losers. In only six of the 15 seasons examined were there more winners than losers. In another six seasons, there were more losers than winners. In the remaining three seasons, the numbers were equal.

Every forecast is for a higher-than-average percentage of winners. All three of the newspapers predict 17 winners this year, or 53.1% of the league. Only once in 15 years has the NFL seen more than half its teams finish 9-7 or better: 2000, when 16 of 31 (51.6%) did it. Sports Illustrated and Sports E-Cyclopedia get as close to the average as possible, yet still go to the high side of it. WhatIfSports goes for an even half of the league finishing above .500. For those interested, the lowest percentage of winners in the past 15 years came in 1999: 11 of 31 teams (35.5%).

All forecasts except one are for a lower-than-average percentage of losers. WhatIfSports hits the 15-year average of 14 losses, or 43.9%. Three sources predict 12 losers, or 37.5%. One picks 11 (34.4%), and we already discussed USAT at 25%. But only once in the past 15 years have fewer than 39% of the league's teams been losers: 1999, when 11 of 31, or 35.5%, finished 7-9 or worse.

(Yes, that's correct about 1999: 11 teams were winners, 11 were losers and nine were 8-8. Now that's parity. Of a sort.)

The last issue I'll bring up related to forecasts is extremes. Generally, if you get 11 or 12 wins, you're good. But if you get 13 wins, you're considered really good. Conversely, if you win only five or six games, you're pretty bad. But you finish 3-13 or worse, you stink. So let's compare how many teams the forecasts say will finish with 13 to 15 wins or 1 to 3 wins, compared with the league averages over the past 15 years:

AVG.SIUSATTampaBuffaloE-CycloWhatIf
13-15 wins2.1120120
1-3 wins2.2202232
Combined4.4322352

There's something a little misleading about this table: No source predicted that any team would finish with one, 14 or 15 wins, and only one source predicted that any team would finish 2-14 (Cleveland, as picked by Sports E-Cyclopedia). So almost all predictions at the extremes are for 3-13 or 13-3. To top it off, all but one forecast lowballed the number of teams at the extremes. The reason for that is obvious: Forecasters can stand to be wrong, but no one wants to go that far out on a limb. I know I wouldn't.

The intention here is not to mock out the forecasts. They're done in fun, and they provide great grist for football talk. But by adhering to certain rules -- evening out wins and losses, sticking to reasonable ratios of winners to losers -- we could devise forecasts that come closer to reality, and thus would prompt even livelier discussion.

No comments: