“Useless Stats”

Though I’m not the most math-savvy sports blogger—I went to Sarah Lawrence—I’m a big fan of well-written statistical analysis. My life as an active and conscious sports fan—as opposed to a guy who just cheers when the team wins—began when I read Moneyball,and since then the most interesting sports writing I’ve read, whether in print or online, has almost all started from the premise that statistics, and the hunt for better and better statistics, are a vital part of how we tell stories about sports. It’s because I’m such a fan of statistical analysis that I want to point out when it could be doing better at telling those stories.

Over at Fangraphs, Jeff Sullivan has an entertaining post in which he shows GIFs from all of Ben Revere’s 19 outfield assists. A glossary for the uninitiated: outfield assists are throws from the outfield that result in a runner being called out (thus, requiring a strong throwing arm, at least in theory); 19 is a respectable number for an outfielder to have after four years in the majors; and Ben Revere is an outfielder with a very weak arm, not at all the kind of player you’d expect to have a respectable number of assists. Using the GIFs to examine how these facts can coexist, Sullivan finds that the 19 assists are split pretty evenly between four very different kinds:

–five of them (4, 5, 7, 16, 17), though technically assists, actually involved Revere throwing to another fielder, who threw to another fielder, and so on until the out was recorded;

–five of them (1, 3, 9, 12, 13) are screwy plays of one kind or another that shouldn’t have been assists at all, either because a runner made an obvious error or because an umpire got a call wrong;

–five of them (2, 8, 10, 11, 15) are conventional assists: Revere throws to a base and the runner is out. Most of these feature slow runners and/or short-distance throws, though one (8), as Sullivan points out, is a legitimately great throw;

–three of them (6, 14, 18) are what Sullivan calls “range-assists”: Revere makes an amazing catch on a ball, one the runners on base didn’t expect; they’ve already taken off running, and by the time they realize he’s caught it they’re too far off the base to have any hope of getting back, so Revere can double them off with even a below-average throw. (One could possibly put 12 in this category rather than in “runner error” since Revere had to cover a lot of ground to make the catch, but the runner has no business being that far off of second base on a ball hit that high, that close to the base.)

So I find this all really interesting, and if you’re still reading I suppose you do too. The trouble is, this post was introduced to me on Twitter this way:

Petriello’s a really smart baseball blogger (though a Dodger fan), and it’s clear to see what his point is here: Sullivan’s post does for sure show a number of problems with using assists as a measure of outfielder skill, especially (as it’s often thought) as a measure of a throwing arm. Particularly important in this respect are what Sullivan calls “tertiary assists,” like the one that goes outfield to cutoff man to pitcher to third base. Whenever a stat is regularly not measuring what fans assume it measures—in this case, an outfielder throwing the runner out directly—it’s going to mislead fans about what story its numbers are telling, which is the opposite of what stats are supposed to do.

Still, I find regrettable the urge to label stats “useless” or “meaningless” or “junk” based on demonstrations like this. In my experience it’s a pretty common reaction, and several commenters on the Sullivan post took the same cue. “[O]utfield assists, much like batting average, are a terrible way to evaluate a player,” said one; “I no longer put any stock in this stat,” said another. Again, my objection is not that these views are wrong; it’s that—and here my Sarah Lawrence background will be obvious—they do not produce interesting discussion. In fact, they’re a way of closing down discussion, of striking an item from the accepted lexicon. The next time someone on the Internet cites outfield assists as a way to evaluate a player under discussion, someone else will be there to cite this link to them as a shorthand for “we don’t talk about outfield assists.” (This may have already happened.) If your job is to work out the player’s value as precisely as possible—and that is a lot of people’s literal job, though maybe not as many as think that’s their job—then this kind of reduction is efficient and sensible. If your job, or at least your hobby, is to have entertaining conversations about the sport, I would submit that it’s regrettable: it closes down avenues for the very goal we’re trying to achieve.

This may sound ridiculously sentimentalist, as though I’m saying that since we always used to talk about outfield assists it would be a shame for us to stop—guarding the canon, basically. But that’s not what I’m saying. Instead, I would agree with Sullivan, who in his post said (though not directly about outfield assists) “there’s value in everything”: even the statistics that are known to be wrong can point out interesting interesting things about the game, sometimes precisely due to the flaws that make them wrong. I want my first reaction to learning about a flaw in a statistic to not be “Well, chuck that one on the trash heap, then,” but rather, “Huh: how did that happen? How did it fool us? Does the stat tell any stories other than the one we thought it told?”

This sort of debate has been going on basically as long as there have been sabermetricians; Bill James observed it back in 1982. “There is no category of statistical information extant which some sabermetrician has not examined in depth, found wanting in one way or another, and so declared to be meaningless,” he wrote in that year’s Baseball Abstract:

All baseball statistics are meaningful; all incorporate illusions. … All individual statistics in baseball are subject to a wide variety of outside and irrelevant influences. There is a great difference between hitting forty home runs in Atlanta [a hitter’s park at that time] and hitting forty home runs in the Astrodome [a pitcher’s park], but that does not make home run totals meaningless. There is a great difference between winning sixteen games for the Yankees and winning sixteen for Seattle, but that does not make won-lost records meaningless. There is a great difference between hitting .300 in Fenway Park and hitting .300 in Anaheim, but that does not make batting averages meaningless. We can sort through all of these things and still see clearly who is and is not a good pitcher or hitter. We can do the same with fielding statistics; it all begins with accepting that there is a difference between being subject to illusions and being meaningless. (reprinted in This Time Let’s Not Eat the Bones; emphasis added)

Thirty years later, won-lost records and (to a lesser extent) batting average are widely perceived, in the statistical community, as meaningless; so is range factor, the stat Bill James was defending in the essay excerpted above. For all I know, James might not even feel this way anymore—he might be happy to dismiss these and many more stats as meaningless. I’m not citing Bill James here because he has authoritative clout, but simply because I think he expresses something important: if we throw out every stat with glaring flaws, we’re going to miss out on some meaning as well. All stats are not created equal—even I am hard-pressed to name a useful observation we can make from pitcher wins and losses—but the divide between Useful and Useless is nowhere near as clean as people want to claim.

So, reading Sullivan’s post and watching the GIFs, the flaws in the outfield assist stat did help me consider something I hadn’t considered before: a great defensive play may be worth more than one out, since the runners will sometimes get caught off the base under the impression that the outfielder has no chance at all on the ball. That’s something that never occurred to me and that I don’t remember ever seeing examined by other writers. If there is an effect there, and depending on how large the effect is, it would influence the way we looked at outfielder range, and at the relationship between range and throwing arm.

Of course, it could very well be an illusion; three of these sorts of assists from one player is barely even an anecdote. The circumstances (a hard hit ball to the outfield with runners on base, and particularly runners who are for whatever reason overconfident that the play is impossible) might be specific enough to make this kind of play too rare to have much of an effect, or at least too rare to study. But it could be something to look into, for those with a head for numbers; for liberal-arts grads like me, it’s something to think about, something to talk about. And that only sounds unimpressive if thinking and talking about baseball isn’t the reason you’re seeking out these kinds of conversations in the first place.

Image credit: “Jamie Burke Slides” by SD Dirk. Licensed under CC BY 2.0 via Wikimedia Commons.

Advertisements

2 thoughts on ““Useless Stats”

  1. I think the smartest and most thoughtful statheads really and truly believe that every stat is meaningful. Or they should, at any rate. I feel like RBI is possibly the best example (either that or pitcher wins and losses) – it’s actually a pretty fascinating and telling statistic. The problem is, of course, is that its used poorly, as a definitive assessment of a players’ worth, rather than an illustration of a player’s offensive surroundings. I can’t remember where I read this (I think the awesome books “Baseball Between the Numbers” maybe?), but RBI are worthwhile at their extremes (you can’t have a 140-RBI season and not be a great player), but there’s this huge muddy range in the middle where there is so much context left out you just can’t draw any conclusions without digging a lot deeper. And this is why I personally fell in love with Moneyball/sabermetrics/that type of fandom: there’s always a story behind the numbers. These Ben Revere outfield assists are a great example too – a weak-armed dude racks up a bunch of strong-armed numbers. Yes, it reveals a flaw in the stat; but even more, it’s fascinating. In that same book I mentioned above there’s a whole part about the worst 100-RBI season ever, by Ruben Sierra. There’s always a story, even with “junk” stats or stats perceived to be meaningless.

    Like

  2. Pingback: Hot dogs, sandwiches, and having a catch: The role of nonsense debates in baseball Twitter | The Spiel

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s