Numbers and advanced stats, why use them?
Thoughts on advanced composite metrics (BPM and RAPM and etc), the value of stats that tell objective truths, contextualizing those objective truths, and more.
This blog is called The Golden State Warriors by the Numbers, so I’m writing a brief piece about how some of these numbers work, why I use certain metrics, and which ones I place value on.
I use a lot of advanced metrics in my pieces despite not fully understanding how some of these metrics work. I’m actually quite awful at math and I still have frequent nightmares of being told I can’t graduate college until I’ve completed a high-level calculus class that I’m certain to fail. I’m 28, by the way. Will those nightmares ever end? I digress.
If you’re like me and you’re bad at math but like basketball and its statistics, this article might resonate with you. If you like basketball and don’t understand or fully trust advanced stats, this article is written for you.
Because I’m bad at math, I don’t fully understand or care to do the legwork to fully make sense of catch-all impact stats like RAPM, EPM, RAPTOR, or BPM. People smarter than I have written about that at length. Here are some articles explaining some of them at length. A long time ago, somebody on the player comparisons board of RealGM defended those metrics in a pretty rudimentary way that made sense to me — they are best at capturing how well a player performs in their role, but they are not an estimation of the best players in the NBA. You’ll see a lot of people claim the latter as they disparage advanced metrics. Mike Muscala ranks #19 in EPM, an advanced stat skeptic might say. You get the picture.
With that said, I still take the catch-all composite metrics with a slight grain of salt. If EPM has somebody higher than RAPTOR, I don’t take that as gospel. My view of it is that the truth is somewhere in the middle of all these stats. You won’t find huge discrepancies in how certain players are ranked within these catch-all metrics. Maybe that means there’s some level of groupthink going on in how those metrics are measured. There are some obvious flaws with some of these metrics. This Twitter thread from April 2022 was eye-opening:
I’ll highlight some of the key findings from that thread below:
In September of 2021, Hoopshype published a fascinating article about how much weight front offices put into the publicly-available advanced composite metrics. Here’s a helluva quote:
“I don’t really use any [advanced composite metrics],” said one executive, who is the president of basketball operations for a team in the Eastern Conference. “They are all pretty bad.”
Others were less critical but felt that while all-in-one composite metrics are constantly getting better, the future of analytics is headed away from these measurements altogether.
“If I could add a wrinkle to your story, it would be that all-in-one stats are overused – that the next phase of basketball analytics is all about context-dependent numbers,” said another front office member from the Western Conference. “That would be the most honest quote I could give.”
Per Hoopshype’s survey of 30 people with experience in NBA media or front offices, Daily Plus Minus’s DARKO stat is the one taken most seriously.
DARKO also accounts for time series and sample size using the complex methods of “exponential decay” and “Kalman filters” to take all historical data into account.
“This is a huge breakthrough because when using possession-level data, it can be tricky to tease out the signal from the noise and oftentimes one season of data is not enough to ensure the proper accounting,” explained Jez, the former head of analytics for the Jazz. “Medvedovsky has solved for this with DPM, allowing him to confidently say when a players’ improvement is more signal than noise.”
The stat that I like most is EPM — don’t ask me to give you reasons for why I like it, it’s a vibes thing, by which I mean it rated Jordan Poole in the top 20 of its offensive players during March of his sophomore season — and is ranked 2nd in Hoopshype’s survey.
Meanwhile, like the aforementioned RPM and RAPTOR, Snarr’s EPM is a hybrid model that uses player-tracking data as part of its evaluation.
We asked Steve Ilardi, co-creator of ESPN’s Real Plus-Minus who has worked as a consultant for NBA teams including the Phoenix Suns, what he thought about the metric.
“[EPM] does a superb job of reducing the high noise level inherent in basic RAPM modeling estimates by integrating Bayesian priors derived not just from box score stats (a la BPM) but also from player tracking metrics,” said Ilardi, during his recent conversation with HoopsHype. “[EPM] has gone one step beyond the RPM estimates that Jerry Engelmann and I developed for ESPN, as we began that project without access to player tracking metrics.”
“I regard EPM as the obvious gold standard of all-in-one metrics,” added Ilardi.
I don’t understand the word gibberish about Bayesian priors, but I like the incorporation of tracking stats into EPM. My internet friend. FNQ, who I interviewed about new-age tracking metrics that he works with, says that most of the composite metrics are already out of date and that more rigorously vetted tracking data is the future of advanced stats.
So this is all to say, I list some of these maligned metrics in my season-in-review posts knowing fully well that those metrics all have blind spots, some of which are significant enough to rate Kyrie Irving as a high-level defender if he is categorized as a big man rather than a point guard. Why do I use them then? Well, I think it’s interesting to view these catch-all impact stats against the more tangible stats, ones that tell you verifiable truths: EFG%, net ratings, points-per-possession (PPP) stats, and tracking stats.
I’d be a fool to treat any of the stats I use in my writing as gospel. Blindly referring to the stats without attempting to contextualize them against the game film, which tells you the why of objective stats, is imperfect analysis. Take, for example, the curious case of rookie-year James Wiseman’s low FG% as a roll-man on the pick-and-roll. Wiseman shot only 52.5% from the field as the roll man and had only a 1.12 PPP on that play type. Wiseman shot worse from the field as the roll man than Juan Toscano-Anderson but was more than 0.2 PPP better than JTA on this play type. How does that happen? Well, JTA only had a score frequency of 40% as the roll man and a TOV% (turnover percentage) of 35%. This is to say, Juan was nearly as likely to turn the ball over as he was to score as the roll man. Maybe you’re starting to remember the various possessions where Juan — bless his heart — would try and make a homerun pass to a shooter or secondary cutter. Context matters.
Let’s go back to Wiseman’s 52.5% FG as the roll man. How does a player so large and so athletic shoot have a field goal percentage 16.3% lower than 2020-21 Kevon Looney? Well, here’s another stat that further informs Wiseman’s FG% as the roll man: James Wiseman shot 75.4% from the restricted area as a rookie and only 31.3% in the non-restricted area paint. James Wiseman took 61 shots on the roll as a rookie. It’s probably a good bet to say that a significant amount of those shots came outside of the restricted area. Now, why might that be? The best guess I had was the Warriors’ floor spacing.
James Wiseman played 1174 possessions of non-garbage time basketball with Steph Curry last season, per Cleaning the Glass. Only 256 of those came without Kelly Oubre Jr. The Warriors’ net rating in the Steph/Wiseman minutes without Oubre was -3.8. Not great, but still much better than the -10.1 net rating of the Steph/Oubre/Wiseman trio. When I run those numbers for Poole/Wiseman with and without Oubre, we get a similar result. Poole/Oubre/Wiseman was an atrocious -23.3 in its 190 possessions together. Poole/Wiseman without Oubre was... positive to the tune of a +12.4 net rating! Mind you, Poole/Wiseman without Oubre only had an ORTG (points scored per 100 possession) of 106.7, which was about six points below league average in 2021. But, believe it or not, Poole/Oubre/Wiseman had the slightly better ORTG of 107.4.
If we go back to looking at Wiseman’s minutes with Steph Curry, those 256 non-Oubre possessions had a +113.7 ORTG. With Oubre on the court, that number crashes to 99.9. What happens if we flip this experiment for Steph/Oubre without Wiseman? A +2.6 net rating and 113.8 ORTG. This is all to say, player interaction and fits play a huge role in how a team’s offense will work. My general theory here is that the Oubre/Wiseman fit — a combo we saw a lot in the Warriors’ starting lineup — hurt the Warriors’ floor spacing, which made it harder for James Wiseman and other Warriors to get into the restricted area, which would help explain his low FG% as the roll man. I’m feeling good about that theory. I feel better about it when I see that the Warriors’ free-throw rate (FT rate% is explained here), is 19.0 for Steph/Wiseman without Oubre and 15.4% for Steph/Oubre/Wiseman.
I’ve burnt a lot of words on Wiseman’s FG% as the roll man to demonstrate the value of not viewing stats in a vacuum. The stats that provide tangible, verifiable truths — FG%, ORTG, net rating, FT rate, and so forth — can often inform each other. If you want to know why a stat comes out a certain way, you need to watch the games because nothing in basketball happens in a vacuum. That’s why I write so often about lineup stats. Even those can be deceptive in small-sample sizes! In a meaningless early-season game against the New Orleans Pelicans, the Warriors ran the Curry/Wiggins pick-and-roll several times in the second quarter and created open shot after shot. But, the Warriors missed 10 straight shots from deep that quarter and so a lineup that did all the right things in its stint together — Curry/Lee/Wiggins/OPJ/Green — finished the week with a -79.1 net rating in 6 minutes that week. Shit happens, which is why I go back to the film to understand the why of it.
For those of you curious, I’ve linked here to NBA Stuffer’s Analytics 101 primer. This primer links to pretty much every advanced stat and acronym I can think of, including most of the ones that I rely on in my season-in-review posts (USG%, EFG%, TS%, ORTG, DRTG, EPM, WS/48, BPM, and so forth). I’ll let the smart people explain how these metrics are calculated and in the meantime, I’ll be happy to keep referring to them because having more information is fun! I’ll keep prioritizing the tangible, objective stats in my analysis — EFG%, ORTG, net ratings, PPP — because I like trying to make sense of them by contextualizing them against each other and using the Chad eyeball test.
Hopefully, this little post gives you some context and food for thought about how I use advanced metrics and publicly available stats, data, and other info.