The Sabermetric Revolution: Assessing the Growth of Analytics in Baseball

7. Estimating the Impact of Sabermetrics

7 Estimating the Impact of Sabermetrics Sabermetricians measure performance—mostly performance on the field, but also in the dugout and in the front office. They seek to inform us through new metrics and analysis what produces wins and profits. In this chapter, we turn the tables by endeavoring to measure the output of sabermetricians. In Chapter 1, we expressed skepticism about the story told by Michael Lewis in Moneyball, or at least about the details of that story. Lewis may have missed a few basic points and misrepresented several others, but that doesn’t mean that the underlying message was wrong. If Moneyball was nothing more than an intriguing fable, it is unlikely that sabermetrics would have spread like wildfire throughout team front offices, as it did in the ensuing years, and as we documented in Chapter 2. Certain sabermetric insights, whether they originated in the work of F. C. Lane, Allan Roth, George Lindsay, Earnshaw Cook, Pete Palmer, Bill James, Vörös McCracken, Tom Tippett, or others, are of indisputable value. There is, for instance, no rational reason to think that batting average is a greater contributor to wins than on-base percentage, that fielding percentage is more important than DER, or that ERA is a more meaningful indicator of future pitching prowess than FIP. Indeed, consider the following statistical evidence. One way to parse the relative importance of the saber-inspired metrics of OBP, DER, and FIP versus the traditional metrics of BA, FPCT, and ERA is to run a regression of win percentage on each set of three variables. We did this for the years 1985 through 2011 and the results were clear. For model (1) below, the coefficient 116 Chapter 7 of determination, or R2 , is 0.80. That is, 80 percent of the variance of win percentage is explained by the variance of OBP, DER and FIP. Conversely, the R2 for model (2) is 0.69. (1) WPCT = f(OBP, DER, FIP) (2) WPCT = f(BA, FPCT, ERA) Thus, the saber-inspired metrics explain 11 percent more of the variance in win percentage than do the conventional metrics. Other things being equal, this suggests that a GM making use of saber-inspired metrics will have an advantage of 11 percentage points toward putting together a winning team, as opposed to a non-saber-inspired GM. But this is easier said than done, as there are many obstacles to move from saber-inspired metrics to building a better team. First, sabermetrics itself is a moving target, since the theory and practice have evolved considerably over the past twenty years. On many issues, there is no consensus as to what the correct sabermetric interpretation even is. For example, the debate about the value of stolen bases continues to this day. Most people who have studied the issue have found that the cost of the out lost by getting caught stealing is roughly two times as large as the value gained by swiping a base. This insight led many teams, most notably the A’s and Red Sox, who were purported adherents of sabermetrics, to become extremely conservative on the basepaths. In fact, among the last twenty team-seasons with the fewest stolen bases, half belong to the A’s and Red Sox, with no other franchise appearing more than once on the list. However, suppose that you were Tampa Bay, and that you already had Carl Crawford on your team. You knew that Crawford stole bases with a historically high success rate (approaching 90 percent), and so you were convinced that by any reasonable accounting, his baserunning would make a positive contribution to your offense. Should you discourage him from stealing bases? Of course not. Thus, while the prevalence of stolen bases is not a perfect metric for estimating sabermetric intensity, it does produce several notable true positives in this context. Second, sometimes the metric in question is easy to identify at the team Estimating the Impact of Sabermetrics 117 level, but difficult to identify at the player level. For instance, DER, or defensive efficiency rating,1 is roughly equal to 1 ‒ BABIP, where BABIP is batting average for balls is play. Hence, DER measures the ability of the team in the field to convert a ball put in play (i.e., not a home run, not a walk, and not a strike out) into an out. This measure is broader than the conventional...