I'm late in the game on this one, but Malcolm Gladwell is in full excitable boy mode in his latest New Yorker piece
, a review of a new book purported to be the Moneyball
In the 2000-01 season, [Allen Iverson] finished first in the league in scoring and steals, led his team to the second-best record in the league, and was named, by the country's sportswriters and broadcasters, basketball's Most Valuable Player.
But how do we know that we're watching a great player?
In "The Wages of Wins" (Stanford; $29.95), the economists David J. Berri, Martin B. Schmidt, and Stacey L. Brook set out to solve the Iverson problem. Weighing the relative value of fouls, rebounds, shots taken, turnovers, and the like, they've created an algorithm that, they argue, comes closer than any previous statistical measure to capturing the true value of a basketball player. The algorithm yields what they call a Win Score, because it expresses a player's worth as the number of wins that his contributions bring to his team.
On average, for his career, [Iverson] has ranked a hundred and sixteenth.
Great opening--Iverson is overvalued, much like Mark McGwire and Sammy Sosa in baseball, and a basketball Billy Beane (general manager of the Oakland Athletics and the hero of Moneyball
) could use that knowledge to put together a cheap team of scrappers to take on the big guys. Please, someone kick out Doc Rivers and make this book head coach of the Celtics.
But when Gladwell gets excited, he screws up his metaphors:
[How well a given individual performs is] an easier question to answer when it comes to, say, golf or tennis, where players compete against one another, under similar circumstances, week after week. Nobody would dispute that Roger Federer is the world's best tennis player.
Most tasks that professionals perform, though, are surprisingly hard to evaluate. Suppose that we wanted to measure something in the real world, like the relative skill of New York City's heart surgeons... recovery time is a function as well of how a patient is treated in the intensive-care unit, which reflects the capabilities not just of the doctor but of the nurses in the I.C.U. So now we have to adjust for nurse quality in our assessment of surgeon quality. We'd also better adjust for how sick the patients were in the first place, and since well-regarded surgeons often treat the most difficult cases, the best surgeons might well have the poorest patient recovery rates. In order to measure something you thought was fairly straightforward, you really have to take into account a series of things that aren't so straightforward. [emph added]
I assumed, reading this, that Gladwell was leading to a discussion of how to separate an individual basketball player's performance from the context of his team, looking perhaps at how well the team did when he was benched or injured, and how well the teams he has played on do before, during, and after his time as a member of their club.
Not so; the "series of things that aren't so straightforward", in basketball, can supposedly by accounted for quite easily by the player's own basic statistics, as weighted by the "win score" formula, which the book's authors post on their blog:
Points + Rebounds + Steals + ½Assists + ½Blocked Shots – Field Goal Attempts – Turnovers - ½Free Throw Attempts - ½Personal Fouls
Nevermind the relative performance of other employees and the prior condition of the patients. And note that this is not an algorithm at all but a formula, making Gladwell's zealous oversell even more cringe-inducing. In typical fashion, Gladwell is getting ahead of the very ideas he's championing.
On the book's authors' blog, they admit the drawbacks of Win Score:
To get at Wins Produced you have to use the exact values, and make a few adjustments – such as adjusting for position played – which we note in the book. Still, Win Score is sufficient to give you a quick snapshot of a player's performance. And it is especially useful if you wish to know if a player is playing better or worse than he did before. [emph added again]
Gladwell, besides confusing win score with wins produced, does not make such disclaimers about either.
The limitations of this, and any, simple formula for sports evaluation are especially clear in the authors' "Quarterback Score" formula:
Yards - (3 x number of plays) – (50 x turnovers)
Certainly a mediocre quarterback with great receivers could compete in such a score with a great quarterback with poor receivers.
A comment on the author's blog post elaborates what an incomplete measurement of a player's value either formula is:
As a former basketball player, one of my strengths was to get the other team's "best player" to foul me. I was even skilled enough to sacrifice a lay-up to draw a foul and then hit two foul shots. By the fourth quarter, the team's star would either be on the bench or super-cautious defensively, given us a tremendous late game advantage. I feel like this contribution lead to a weaken defense and would have given me more Win Score than your formula would give me, since there are minute factors like this that could come in play even in the NBA.
Also being a "smart player" comes into play when fouling a player. I would like to think there are players, like myself, that make "smart fouls" that save 2 points more than it gives 2 points to the other team. ( i.e. stopping a fast break at half-court.) This could sway the Win Score too.
Not to mention reputation too. For instance, Shaq... teams probably shoot more inaccurate outside shots when he's in the game which would give his team an advantage, because they are less likely to drive the lane. This also would decrease Shaq's opportunities at blocked shots that could raise his Win Score number too. I'm sure for a player like Allen Iverson, an opposing coach would put his best defensive player in the game to cover him (sacrificing the regular starter who's more offensively skilled.) The affects of this would be tremendous as well.
The authors acknowledge these drawbacks and admit that the formula's can err widely with individuals, even as it settles down and decently approximates the quality of the aggregated players on the team. Gladwell does not acknowledge this.
Gladwell writes that, among other surprises,
Jermaine O'Neal, a center for the Indiana Pacers, finished third in the Most Valuable Player voting in 2004. His Win Score that year put him forty-fourth in the league.
The Win Score algorithm suggests that Ray Allen has had nearly as good a career as Kobe Bryant, whom many consider the top player in the game, and that the journeyman forward Jerome Williams was actually among the strongest players of his generation.
I calculated the win score of fan favorite Jerome "Junkyard Dog" Williams: he has a career win score of 6.3. Jermaine O'Neal is certainly overrated, and Gladwell is right that he has never been close to MVP, but his career win score edges out JYD's at 6.5, and (using Berri & co.'s formula) his win score for the 2003-4 season Gladwell mentions was 8.4. To put this in context, in Jason Kidd's excellent 2004-5 season, the single best performance in terms of win score that Gladwell cites, his win score was 10.1. Considering his was Gladwell's cherry-picked example, O'Neal's 8.4 doesn't look so bad.
I assume Gladwell's 20 games number for Kidd is a confusion with "wins produced", which the authors discuss on their blog.
For perspective, the best career win score I could find for a player who entered the NBA within 1 year of O'Neal and Williams was Tim Duncan's, 12.9. While we're at it, Iverson's career win score is 5.4, low indeed for a former MVP, but not as low as it was in the 2000-2001 season, when it was 4.9 due mostly to his poor shooting percentage and his high number of free throws.
At this point, if you're not wondering what Michael Jordan's win score was, you may as well become a monk. His career score was 10.7. Yes, this includes the various comeback periods when he was less than 100%, but his numbers in those seasons weren't as bad as I thought. I understand that wins produced provides adjustments for position (though where that leaves position-bending players like Scottie Pippen and Le Bron James, I don't know), but does his high win score mean Duncan is on par with Jordan?
Gladwell's article points out the mistake of assessing player performance using very limited information--watching them play and looking at their scoring--but he doesn't ask how we evaluate panacea formulas for sports performance, which are nothing new. John Hollinger, basketball's closest thing to baseball's pioneering Bill James, made up a complex formula called "player efficiency rating" that is very influential, but seems to mesh with basketball fans' evaluations of ball-hogging stars like Allen Iverson and Kobe Bryant better than Berri & co.'s win score does. Berri & co. call their book an attempt to both simplify and improve PER (much needed, since there are mathematical symbols in the PER formula I don't remember learning in college).
Gladwell is also short on details of who else is working in the field. As usual, his subjects are islands of insight and wisdom, and you can't blame a CEO for concluding that the best way to gain the wisdom of Gladwell's maverick subjects is probably to pay him a huge honararium to speak at the annual company convention. Gladwell should note that as in baseball, the statistics nerds with laptops are alreadly being hired and given power to test their theories for real teams; mentioning these up-and-comers might help spread the management consultant largesse.
It's easy to poke holes in a busy writer's short book review, but there is one paragraph that fully deserves evisceration:
Most egregious is the story of a young guard for the Chicago Bulls named Ben Gordon. Last season, Gordon finished second in the Rookie of the Year voting and was named the league's top "sixth man"—that is, the best non-starter—because he averaged an impressive 15.1 points per game in limited playing time. But Gordon rebounds less than he should, turns over the ball frequently, and makes such a low percentage of his shots that, of the N.B.A.'s top thirty-three scorers—that is, players who score at least one point for every two minutes on the floor—Gordon's Win Score ranked him dead last.
What a peculiar example. Of course Gordon is dead last--he's a sixth man! "Win score" values absolute
numbers of points, rebounds, etc. much more than it does the relative rate of these things. Almost all of the other 33 top scorers are surely starters who play 20-30 more minutes per game than Gordon does, since surely even the most curmudgeonly, old-school coach will start his most fast-scoring players in each position.
If all teams were equal, the best sixth man in basketball would rank 161st in win score, since there are 30 teams with five starters each, and I imagine that if Gordon's rank was worse than 161st, Gladwell would tell us. 'Worse among the top 33 selected by criteria X' sounds bad, but if Gordon were 33rd in the league in win score per minute played
, which is the best way I can think of offhand for a sixth man to be measured, he would be not only a shoo-in for sixth man of the year, but perhaps the best sixth men of all time.
Gladwell's example is so convoluted as to be disingenuous: if you select the arbitrary number of 33 and the arbitrary measurement of scoring rate, which comes up no where else in the article (instead than win score, which is the subject of the article), and you list the top players according to that measurement, and then you rearrange the players according to win score, then
Gordon is "dead last". Quite a production.