Hi there. It's been a little while. (I think it's been two, maybe three site redesigns since I've been up here. Neat.)
Nico asked me if I could explain how advanced defensive statistics work and how they're created. For those who don't know, I'm a former writer here at AN, now working at Baseball Info Solutions, the company behind DRS. There’s a lot of confusion and misinformation out there, so I’d love the opportunity to set it straight. I’ll outline the metric I’m most familiar with, DRS (Defensive Runs Saved), and touch on a few comparisons with UZR (Ultimate Zone Rating) along the way.
I’ll try to keep it simple and stay on a conceptual level, with as little math and numbers as possible.
Plus/Minus
When it really comes down to it, when you strip away everything and whittle defense down to its core, what separates a good fielder from a bad one? The ability to make more plays and record more outs.
So if you were building a defensive metric, the first thing you’d need to figure out is how many plays a fielder made, and how many plays the average fielder at his position would have made, given the exact same distribution of batted balls. Josh Reddick has a ton of range in right field, so he’s going to get to balls most other right fielders can’t reach. Summing up the amount of plays made, as compared to the average fielder, is the heart of both DRS and UZR.
But we can’t just rely on whole numbers and simple yes/no data, either. Let’s say a third baseman ranges two steps to his left and scoops up a grounder. Would the average third baseman have converted that ball into an out? It’s not a black-and-white answer, and we can’t just give him a point if the answer is no and give him nothing for a yes. There isn’t a line in the dirt, where everything on one side is 100% reachable, and everything on the other side is 100% not. We need to award partial credit. To do this, we break up all batted balls and categorize them in terms of their location on the field and the time it took the ball to get there. Let’s use a real-world example.
(Click to see the gif.)
This was from June 30th of this year, in the top of the 7th inning against Allen Craig (2:12:30 on MLB.TV, for the curious). Josh Donaldson quickly dives over to the left and stabs at the ball with his glove, just in time to stand up and throw for the out. It’s a tough play. A really tough play. Let’s say that groundballs hit to that zone of the field with that batted ball speed are only turned into outs 20% of the time. Since Donaldson made that play, we give him 0.80 points, which comes from 100% minus the 20% from the successful attempts. What does that mean? In that inning, Josh Donaldson made 0.80 more plays than the average third baseman.
Fielders even get a little positive credit on the easy plays, because no play is made absolutely 100% of the time without a single mistake. Let’s say a relatively slow 40 mph groundball hit right to the third baseman is fielded for an out 98% of the time. If Donaldson turns the out, he’ll get 0.02 points. If he lets it skitter underneath his glove, he’ll be docked -0.98 points.
That’s it. Seriously. Advanced defensive metrics can seem dense and bewilderingly complicated at times, but really, this is almost all there is to it. Understand this, and you understand DRS and UZR.
The best part about this approach is that it ignores things like dives, spins, or any of that cosmetic Jeter-y stuff. Convert that tough play while making it look boring and routine? 0.80 points. Dive, spin, and throw while jumping, just barely getting the runner out? 0.80 points. Unlike the eye test, nobody subconsciously gets extra credit for making things look cool on Sportscenter.
If you keep a running total of every play for every fielder over the whole season, adding up the plays where he made an out and subtracting plays in exactly the same way when an out wasn’t recorded, you get a number, which DRS calls Basic Plus/Minus. These numbers are then multiplied by a factor for each location/time zone that accounts for extra base hits. An outfielder who allows two balls to drop in front of him should be docked fewer points than an outfielder who allows two balls to drop behind him, since if all else is equal, the former are almost always less-damaging singles, while the latter often go for extra bases. By multiplying by this extra-base factor, we’ve just converted our plays above average number to bases above average. This is what DRS calls Enhanced Plus/Minus.
Translating to runs
This is almost completely identical to the way wOBA or any other offensive stat is translated to runs, but I’ll outline it here, because it’s important.
There’s a concept in sabermetrics called the 24 base/out states. It may sound difficult but it’s not, I promise. There are eight ways the bases can be configured: bases empty, guy on first, guy on second, guy on third, guys on first and second, guys on first and third, guys on second and third, and bases loaded. Trumpet players, that probably sounded a bit familiar. Eight base states multiplied by three out states (zero, one, or two) gives us 24 base/out states that describe every possible situation within an inning.
Now, if we look through baseball history, we can find the amount of runs that have scored from any one of the base/out states through to the end of the inning. If we average all of that data and group it by the base/out state, we get what is called run expectancy, because it tells us the value of each of the base/out states in terms of runs, without having to rely on what comes next. For example, having the bases loaded with one out is worth 1.631 runs on average, according to this run matrix that was distilled from every single in-game baseball event from 1993 to 2010.
Since we know the value of all of these states, we can easily compare them. Let’s say there’s a runner on first with one out, and the batter hits a hard grounder to the shortstop. Jed Lowrie dives, can’t get it, and the ball winds up in left field. Now there are runners at the corners, still with one out. According to this run expectancy matrix, the difference between the previous situation and the current situation is +0.649 runs, which means that the botched play was worth exactly the opposite: -0.649 runs.
To make a long story and a lot of math short, by averaging all of this together, we can find a conversion factor for each defensive position, so that we can go from bases saved above average to runs saved above average.
You can find DRS’s Plus/Minus Runs Saved at Fangraphs under the "rPM" column. For most fielders, this makes up the bulk of their final DRS score. UZR’s equivalent is on the same table at Fangraphs, and is labeled as "RngR".
Other stuff
What else goes into DRS aside from Plus/Minus? There’s the Good Fielding Plays/Defensive Misplays (GFP/DME) system, where BIS’s video scouts flag good and bad plays and categorize them into predefined descriptive groups. These categories include events like when an infielder has to settle for the out at first, because he bobbled the ball and lost the lead runner, or when a fielder doesn’t cleanly field a ball, but traps it near him and prevents runners from advancing further. The categories that don’t double-count things already in Plus/Minus are converted to runs above/below average in a similar way, then added in. Stuff like this is counted in the "rGFP" column on Fangraphs.
There’s also the system which awards credit for turning double plays. Outfielder arms are measured. Catchers and pitchers are given credit for controlling the running game. It’s probably easier to list all of the inputs into DRS for each group of positions.
Corner Infielders: Plus/Minus, double plays, bunt fielding, GFP/DME.
Middle Infielders: Plus/Minus, double plays, GFP/DME
Outfielders: Plus/Minus, outfield throws, GFP/DME
Catchers: Pitcher handling, baserunning prevention, bunt fielding, GFP/DME
Pitchers: Plus/Minus, baserunning prevention, bunt fielding, GFP/DME
Frequently asked questions
I’ll take questions in the comment section, and if I think they’re useful enough to be highlighted, I’ll reprint them up here and answer them in the body of this thing for reference. For now, I’ll try to stave off some of the more common questions I’ve seen.
How can you talk about runs above/below average for a play when a run didn’t even score in that inning anyway?
In this entire context, we’re talking about theoretical runs that are calculated on average. It’s true, we don’t look at runs that actually come around to score and then dole out credit after the fact. Why? Because if a second baseman botches a play that allows a baserunner, why should we ignore the bad play just because the pitcher struck out the next two batters to escape unscathed? The second baseman still made a bad play, and what happens next is irrelevant. So we calculate everything on average, that way, we know what that single play was worth, isolated from everything else that may have happened.
Simply, what is the difference between DRS and UZR?
While both get their data from BIS and use very similar methods (especially in the Plus/Minus calculation), DRS has batted ball timer data, while UZR does not. The GFP/DME system is also exclusive to DRS, while UZR attempts to measure something similar by adding up errors.
Don’t first basemen get misrepresented because Plus/Minus doesn’t care about scooping ability?
Plus/Minus doesn’t care about scoops, but as a whole, DRS does. When BIS’s video scouts see a first baseman recording an out on a bad throw, they’ll flag it as one of the Good Fielding Plays, which gets factored in. The end result is that the first baseman gets some extra credit, while the infielder who made the bad throw will get docked.
What about shifts? DRS gives fielders credit for positioning, but you can’t blame a second baseman for not making a play when his coach told him to stand ten feet out into right field.
Correct. Both DRS and UZR ignore plays where there was a shift. DRS then circles back around and computes runs above average on shifts separately, on a team-wide basis.
What about first baseman positioning? If they’re holding on runners, doesn’t that limit their range?
Yup. DRS takes out all first baseman plays with a runner on first, then calculates them all separately.
I'm a glutton for punishment. What if I want to read more?
You can find detailed methodologies for DRS here (under the Plus/Minus tab), and for UZR here.
