Reading and Thinking Football

Football, including books thereon and idiosyncratic thinking thereabout

Book Review: The Pro Football Historical Abstract

with 2 comments

Sean Lahman’s The Pro Football Historical Abstract: A Hardcore Fan’s Guide to All-Time Player Rankings should have been right up my alley, the kind of book I should have devoured when it first arrived. It was his attempt to do a version of Bill James’s New Bill James Historical Baseball Abstract, which combined an interesting and valuable method of putting players in historical context with entertaining and informative mini-biographies of players (or so approximately everyone who’s read it claims; I’ve not read it, and my appetite for reading about baseball is now nil). That I started the book, paged through it, and put it down for four and a half years is perhaps the best and simplest review I could do.

Beyond that, though, Lahman’s Abstract has three basic problems:
1. The rankings produce screwy results;
2. The ranking methodology that produces the screwy results is not sufficiently justified;
3. Irrespective of the rankings and the quality of his methodology, Lahman for some to many players fails to properly execute his methodology, invalidating to some extent all of his rankings for those positions.

The Abstract begins with a brief decade-by-decade summary of the NFL. They are, by the description, intended to provide context for the player capsules that follow. In reading them, though, they come out more like less interesting, less good versions of the similar decade summaries in the Pro Football Hall of Fame 50th Anniversary book. Had these been heavier on schemes and the distribution and accumulation of player statistics, this could have been a very useful exercise. How many carries did a team’s lead back normally have in each decade? How many plays were run in a game, and how many of them were runs v. passes? What were turnover rates like? How competitive overall was the league? Were there a couple good teams, or was the league relatively balanced? Heck, just how extensive and how reliable are the statistics we have for this era? More teams playing in domed stadiums and on artificial turf, information that unlike the questions I asked is actually included, does not strike me as nearly as useful.

The heart of the book, though, is the player rankings. The easiest thing to do with rankings is to nitpick them. Why is player X ahead of player Y and behind player Z? I believe it’s a Bill James quote that a new statistical method should 90% confirm and 10% go against what you were already pretty sure you knew. The fact that Steve McNair came out the 15th-best quarterback of all time, ahead of, inter alia, Sammy Baugh, Sid Luckman, Bart Starr, Jim Kelly, Terry Bradshaw, Troy Aikman, Warren Moon, and a bunch of other players does not per se make Lahman’s rating system wrong. Ditto the fact that his methodology for ranking seasons includes Larry Johnson’s 2006 season and does not include either O.J. Simpson’s 1973 or 1975 campaign (which ranked 7th and 1st, respectively, in, for instance, Chase Stuart’s ranking of the most dominant RB seasons in history). That I find these rankings dubious does not mean that they are wrong. It means, rather, than they must be justified. Lahman doesn’t do so.

His base rating is Adjusted Yards. It has some elements that make sense, punishing fumbles lost (a more available indicator than total fumbles), interceptions, and sacks, and giving a bonus for touchdowns. Rushing yards are treated at face value. That Johnson had over 400 carries in 2006 doesn’t matter in Lahman’s ranking of him (to be fair, it doesn’t in Chase’s ranking system, but I don’t love that either). One of the hard questions is how to separate the value of the quarterback and the wide receiver. Lahman’s solution is that giving them both credit for their nominal yardage overweights the passing game, so he splits them up, giving half the yards to the receiver and half the yards to the quarterback. Why a 50-50 split is the right solution to the problem is left as an exercise for the reader. No, really. Here’s your justification:

Now we come to the part that may seem counterintuitive. How do we divvy up credit between quarterbacks and receivers on a passing play? Consider the quandary that existing football statistics create for us. Joe Montana throws a 12-yard pass to Jerry Rice. Montana is credited with 12 passing yards. Rice is credited with 12 receiving yards. Isn’t that double-booking? Seems like Enron-style accounting to me. The net gain is 12 yards, not 24. The quarterback and the receiver should each receive credit for half of the total yards on each passing play.

I do not think I could have made that up if I tried.

Once Lahman has his Adjusted Yards number, he then applies it to each season and gives each player a Q score based on how many Adjusted Yards they had relative to the league leader. The league leader receives 10.0 Q, while other players get a Q based on their Adjusted Yards relative to the league leader. This leads to, at times, counterintuitive results. Take, for instance, Raymond Berry. What was his best season? Looking at his P-F-R page, I would say perhaps 1959, when he led the league in receptions, yards, and touchdown passes, though you could make a case for 1960, when he had more receptions and a better yards per catch number. By Lahman’s scores, it’s actually 1957, when he had a Q of 10.0. 1959 comes out as Berry’s fifth-best season, also behind 1958, 1960, and 1961, when he had 873 receiving yards and no touchdowns. I’m prepared to entertain unconventional claims, but that a receiver had a better season the year he had 80 fewer receiving yards, 10 fewer receiving touchdowns, and 113 fewer of Lahman’s Adjusted Yards than he did the year he led the league in yards, receptions, and receiving touchdowns  is awful hard to understand.

I could live with even a screwy, insufficiently justified ranking system if the individual player capsules were interesting in their own right, or at least told me more about why players were ranked where they are. They aren’t. They’re okay, and I have no particular objections to them, but they are not themselves that entertaining. Having 75 running back capsules and 25 on each of the offensive line and defensive line I can sort of live with, in that it just represents an acceptance of everything that’s terrible about football writing, but that doesn’t mean like it.

If you like more screwy, insufficiently justified ranking systems, he also has one for coaches. Ranking coaches is hard. How much do you value regular season relative to postseason success? Should influence on the rest of the league matter? Are men who coach longer inherently better? Does it matter that more teams make the playoffs than used to, or that the overall quality of the league is higher than it was in the early days of the AFL? Lahman’s system is additive and playoff-heavy.

The fourth part of the book, comprising over 240 pages, is more or less the statistical appendix, where you can see things like Berry having fewer receptions, touchdowns, and Adjusted Yards and a better Q score. Curious about some of his math, I decided to check Steve McNair’s box score. Subjectively, I think McNair only had three very good seasons, 2001-03, with the best of those being the 2003 year he was co-MVP with Peyton Manning. By Q, though, that’s his sixth-best season. His best was instead 2001, when he led the league in Adjusted Yards. He finished second in passing DVOA to Kurt Warner, so I decided to take a look at their numbers that year, to see why McNair had a Q of 10.0 and Warner a Q of 9.1. The answer, unsurprisingly, is in part McNair’s rushing productivity-he had 350 more rushing yards and 10 more rushing touchdowns, which accounts for 450 Adjusted Yards.

The other, more troubling answer is that McNair did not have more Adjusted Yards than Warner in 2001. Lahman tells us he did, but he’s wrong. The problem is fumbles. Lahman for his table and his calculation uses total fumbles, while his formula calls for the use of fumbles lost. Warner had 10 fumbles in 2001, losing 4 of them. McNair had 5, losing 3. Using the correct fumbles lost information, Warner should have a Q score of 10.0 while McNair’s should have been 9.7. I haven’t checked all the tables to see exactly how widespread the problem is, but it appears to exist for at least all modern “skill position” players, and I’m pretty sure it goes back to at least the 1960’s. This isn’t obvious unless you actually calculate the numbers, and I didn’t realize the problem until I started writing this review. I should stress that I believe Lahman’s error is an honest screwup, and the kind of thing that keeps anybody who works with data up nights. Knowing the error, though, I have to regard all of his scores for QB, RB, WR, and TE as somewhere between dubious but probably somewhat close and wrong. Rankings for the other positions are not affected by this error, but it’s more trouble to cross-check them. Can I trust those other rankings, knowing he screwed up the ones that are easy to check?

Before I discovered the flaw noted at the end of my review, I was prepared to issue a tepid non-recommendation. Now, though, I simply cannot countenance any purchases of Abstract.


Written by Tom Gower

February 22, 2013 at 11:00

Posted in Book Reviews

2 Responses

Subscribe to comments with RSS.

  1. You should really read the new bill james historical baseball abstract even if you hate baseball. It’s far better then many of the questionable sports books you read(such as this one) and is probably the best single book available on the history of the sport, though a little outdated now. It would also allow you to get even madder at how bad this book is in comparison

    joe football

    March 9, 2013 at 12:53

  2. […] football reads this quarter included Sid Gillman by Josh Katzowitz, The Pro Football Historical Abstract by Sean Lahman, and Football Physics by Timothy Gay. I have reviews of the first two up, and […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: