Measuring How NBA Players Help Their Teams Win
May 30, 2004
I. Introduction
“Good
players lead their teams to wins.” “Lots of players can fill up a stat sheet,
but only the great ones are difference-makers.” “The objective of a basketball game is not to
accumulate points or rebounds or assists, but
to win. What statistic do you have
for that?” When I talk with
knowledgeable basketball people who are skeptical of statistical analysis, I
hear variants on these statements over and over again. The argument is that winning is important and
game statistics are only an imperfect measure of many of the contributions that
players make to winning.
I
very much take this argument to heart.
Basketball is not like baseball, a game structured around repeated
one-on-one contests between pitchers and batters, where the contributions to
winning of any given player can be measured well by individual game statistics. Basketball is much more of a team game, and
as noted by Dean
Oliver, one of the leaders of statistical analysis is basketball, “teamwork is
the element of basketball most difficult to capture in any quantitative sense”
(p. 77). While this argument often is overstated (as
much of this teamwork can be measured
using game statistics), the point still stands.
These limitations have led to new approaches to measuring the value of
basketball players, approaches that make little use of game statistics like
points, rebounds, and assists.
The most common approach is to compute plus/minus ratings that measure how point differentials change when a particular player is in the game versus when he is not. Hockey has used such a plus/minus system for years, but Roland Beech of 82games.com is the first to make these data available for the NBA. The logic of this approach is straightforward; teams should perform better when their good players are playing versus when they are not. The intuitive appeal of this approach has not escaped teams’ attention, and my understanding is that most teams use plus/minus ratings to some extent. However, these “unadjusted” plus/minus ratings do not measure the value of a player per se; they measure the value of the player relative to the players that substitute in for him. In addition, there are differences in the quality of players that players play with and against. A weak starter on a team with exceptionally good starters (relative to bench players) will generally get a very good unadjusted plus/minus rating – regardless of their actual contribution to the team.
Thus,
a better measure of player value would “adjust” these plus/minus ratings to
account for the quality of players that a given player plays with and
against. In addition, it would account
for home court advantage and for clutch time/garbage time play. Thus, unlike in unadjusted plus/minus
ratings, these “adjusted” plus/minus ratings do not reward players simply for
being fortunate to being playing with teammates better than their
opponents. Contributions for individual
players are isolated statistically. In this article, I develop adjusted
plus/minus ratings similar to the
WINVAL ratings designed by Jeff Sagarin and Wayne
Winston.[1] I improve on past
efforts by combining estimates of player value using both pure adjusted
plus/minus ratings and a statistical index derived from these pure adjusted
plus/minus ratings. This hybrid approach leads to player ratings that unlike press accounts of WINVAL
ratings, pass the “laugh test” (p. 181). In addition, the results from this approach
are even less noisy than ratings based on traditional statistical indices
alone.
Using
data from the 2002-03 and 2003-04 seasons (with the latter season being
weighted twice as heavily), I find that Kevin Garnett, Tracy McGrady, Andrei Kirilenko, Tim
Duncan, and Shaquille O’Neal are the five most
effective players in the NBA. Replacing
an average player with one of these five players would result in a team
improving by about 14 points per 100 possessions or a little over 10 points per
game. In other words, in 2003-04
replacing one of the average players on the Orlando Magic with one of these
five players likely would have made them a bit better than the New Jersey Nets
and Memphis Grizzlies.
Perhaps
more importantly, with these adjusted plus/minus ratings I am able to estimate
what game statistics predict better performance on the court; these results
help explain why certain players have such high adjusted plus/minus
ratings. It appears that rebounds are
less valuable than typically assumed and steals, blocks, and avoiding turnovers
are more valuable. It also appears that
having three point shooters on the floor helps teams and that players that can
do it all – score, rebound, and assist – are more valuable than simply the sum
of those game statistics. In addition,
even after accounting for all of those game statistics, players who play more
minutes tend to be more valuable for their teams. This finding suggests that coaches recognize
those contributions to the team that are not measured by game statistics, and
they play those players more minutes.
In
this document I lay out a lot of the details of what I am doing and many of you
may want to skip over those details. If
you just want to see my bottom-line ratings, go to Table 4 or
Table 5.
II. A Discussion of the
Set-up and Results
Here
is the set-up that I use. Every observation is a unit of time in a game where
no substitutions are made. There are more than 60,000 such observations
per year in 2002-03 and 2003-04. With these data I run the following
regression.
(1) MARGIN
= b0 + b1X1 + b2X2 + . . . + bKXK + e, where
MARGIN = 100 * (home team points per possession
– away team points per possession)[2]
X1
= 1 if player 1 is playing at home, = -1 if player 1 is
playing away, = 0 if player 1 is not
playing
XK = 1 if player K is playing at home, =
-1 if player K is playing away, = 0 if player K
is not playing
e = i.i.d. error term
b0 measures the average home court
advantage across all teams
b1 measures the difference between player 1
and the reference players, holding the other players constant
bK measures the difference between player K
and the reference players, holding the other players constant
The reference players are all players playing
less than 250 minutes in both seasons combined. Observations are
weighted by the number of possessions with (1) observations in 2003-04 weighted
twice as heavily as those in 2002-03 and (2) higher weights during crunch
time and lesser (or zero) weights during garbage time.[3]
It is in this regression where the effects of the other players on
the floor are accounted for. The bs in equation (1) measure
the point differential difference (measured per 100 possessions) of the given
player relative to the reference players, holding constant all of the players
that shared the floor with that player (and with the reference players), i.e.
holding the other players constant. What
does this “holding other players constant” mean? Strictly speaking, it means that we can take
a player and surround him with four teammates and five opponents and compare
how that player’s team would do versus how it would do if he was replaced by a
replacement player keeping all of the other players the same. This is what is meant by “holding the other
players constant,” since we can repeat this exercise with any other combination
of other players.
Another
way to think of these bs is that they are
plus/minus statistics adjusted for the other players on the floor. This takes out the effect of a player who is
fortunate to always play with Kevin Garnett or unfortunate enough to always
being matched with rookies or NBDL players.
Table
1 presents the results from equation (1) for the top twenty players among those
playing 250 minutes or more in the 2002-03 and 2003-04 seasons combined. (I normalize these ratings so that the
average player is given a value of zero.)
Kevin Garnett has by far and away the highest “pure” adjusted plus/minus
statistic, being a full 19.3 points per 100 possessions better than the average
player. In addition, this estimate for
Garnett is quite precise with it being statistically significantly different from
most the rest of the players in the top ten.
The rest of the top ten (with the exception of Nenê
and players with high standard errors and less than 1,000 minutes) are among
the top players in the game – Vince Carter, Andrei Kirilenko,
Dirk Nowitski, Tim Duncan, and Shaquille
O’Neal. The next ten contains another
five players among the top players in the NBA – Rasheed
Wallace, Ray Allen, Tracy McGrady, Baron Davis, and
John Stockton.
That
said, there are a number of outliers in the top 20. Six of those nine outliers (Richie Frahm, Jason Hart, Mike
Sweetney, Mickael Pietrus, Earl Watson, and Carlos
Arroyo) have standard errors ranging from 4.3 to 6.3, so these players’ high
ratings could mostly reflect sampling variation – although Frahm’s
rating is so high that despite the high standard error and low number of
minutes, it probably is something more than sampling variation. Three players (Nenê,
Jeff Foster, and Eric Williams) seem to have genuinely quite good ratings that
cannot be explained away by sampling variation.
Foster replaced an All-Star in Brad Miller and his team did not miss a
beat, ending up with the best record in the League. Nenê played major
minutes for a team that improved dramatically in 2003-04, and Eric Williams
played on two teams (
However,
even taking all of that into account, these ratings are quite noisy. Another approach is probably necessary for
these rating to be that useful. Below I
outline that approach which combines these pure adjusted plus/minus ratings
with ratings derived from the relationship between game statistics and these
pure adjusted plus/minus ratings.
(I
also present offensive and defensive ratings that are based on the pure
adjusted plus/minus rating plus an “efficiency” rating that measures how many
points per possession are scored by both teams when a given player is one the
floor. By combining these two measures,
I create offensive and defensive ratings.
However, given that I am using two imprecisely estimated ratings to
arrive at these offensive ratings, I suspect these rating are measured with
quite a bit of error.)
Table 1: Pure Adjusted
Plus/Minus Ratings for the Top 20 Players in 2002-03 and 2003-04
|
Rank |
Name |
Pure Adj. +/- |
Offensive |
Defensive |
Poss. Used |
Offensive Efficiency |
Total Minutes |
||||
|
First |
Last |
Rating |
SE |
Rating |
Rank |
Rating |
Rank |
||||
|
1 |
Kevin |
Garnett |
19.3 |
3.0 |
113.7 |
2 |
94.4 |
15 |
28% |
108 |
6,553 |
|
2 |
Richie |
Frahm |
17.3 |
6.3 |
114.0 |
1 |
96.7 |
54 |
15% |
126 |
466 |
|
3 |
Nenê |
|
11.9 |
2.7 |
104.3 |
43 |
92.4 |
5 |
18% |
101 |
4,755 |
|
4 |
Vince |
Carter |
11.1 |
2.5 |
108.1 |
9 |
97.0 |
69 |
30% |
101 |
4,255 |
|
5 |
Andrei |
Kirilenko |
11.1 |
2.6 |
108.6 |
8 |
97.5 |
89 |
22% |
106 |
5,108 |
|
6 |
Dirk |
Nowitzki |
10.6 |
2.7 |
109.8 |
5 |
99.2 |
176 |
24% |
115 |
6,033 |
|
7 |
Tim |
|
10.3 |
3.3 |
107.2 |
14 |
96.8 |
59 |
28% |
106 |
5,705 |
|
8 |
Jason |
Hart |
10.1 |
5.6 |
100.7 |
139 |
90.6 |
2 |
15% |
99 |
660 |
|
9 |
Mike |
Sweetney |
10.0 |
5.8 |
106.9 |
17 |
97.0 |
66 |
19% |
104 |
495 |
|
10 |
Shaquille |
O'Neal |
9.9 |
3.0 |
107.8 |
12 |
97.9 |
108 |
27% |
110 |
4,999 |
|
11 |
Rasheed |
Wallace |
9.6 |
2.1 |
105.3 |
27 |
95.7 |
31 |
22% |
105 |
5,073 |
|
12 |
Mickael |
Pietrus |
9.5 |
4.3 |
106.8 |
19 |
97.3 |
81 |
18% |
102 |
748 |
|
13 |
Ray |
Allen |
9.0 |
2.3 |
109.7 |
6 |
100.7 |
248 |
28% |
110 |
5,029 |
|
14 |
|
McGrady |
8.6 |
2.7 |
109.9 |
4 |
101.3 |
277 |
32% |
111 |
5,630 |
|
15 |
Earl |
Watson |
8.1 |
4.5 |
107/0 |
15 |
98.9 |
159 |
21% |
90 |
3,036 |
|
16 |
Jeff |
Foster |
7.7 |
2.7 |
103.7 |
52 |
95.9 |
37 |
14% |
108 |
2,774 |
|
17 |
Baron |
|
7.6 |
2.5 |
104.4 |
40 |
96.8 |
58 |
29% |
100 |
4,575 |
|
18 |
|||||||||||