Draft WAR by General Manager. Beane #1 in Baseball Analysts study. (Giants suck.)

"I have a dream…That 1 day AN will actually realize baseball is played on grass and not paper." May this become "The famous AN ‘I have a dream’ speech."

Right. Clearly you love statistical analysis.

Do you have any criticism of WAR other than you don’t like it? Do you understand why an exceedingly high (or low) BABIP is a sign of luck?

The frustrating thing is that there is a carefully built foundation for these stats, which you apparently take issue with for unknown reasons.

When I hear a sentence like, “Adrian Beltre was worth 3 more wins than Bobby Crosby,” I want to hurl my breakfast, and not just because I heard the name Bobby Crosby.

as it is that I don’t like the behavior of the people who tend to use them the most! But specifically with WAR, an additional problem I have is that “wins” is precisely the “unit of measurement” and I think that’s a fundamentally flawed concept.

I have no problem with assigning a ranking or number value to a player, e.g., saying that “On the 1-10 GNM scale, Hannahan is a 5.5 and Uggla a 4.8, so it might surprise you but you factor in position, defense, OBP, etc. Hannahan is actually the more valuable player.” But the very point of WAR — by its own measure — is that players are individually worth a certain number of wins compared to other individuals. And I think that’s a very poor unit of measurement to use.

GNM stands for “Generically Named Metric,” by the way.

My problem with WAR is exactly like what I described in another comment above. The guys who devise these WAR numbers are very responsible in describing exactly what they’re doing. You can usually read their methodology summary where they say, "I have calculated the following blah blah blah, omitting blah blah and blah, assuming that blah blah blah. These are compiled into a single figure which thus represents blah blah meta-blah. The end result is then scaled against the league average for a number which I call ‘WAR’."

This is exactly as it should be. We can all take a look and see what’s being measured and base our conversations on it accordingly. But that’s not always what happens. What sometimes happens is some joker comes along and says, "I just looked up their WAR’s and the stats prove that Mookie Cabrera is two whole wins better than Beanhead Rodriguez, so ha ha ha, your stupid," possibly accompanied by a cartoon of an ostrich with its head in the sand. That’s what pisses me off.

I have no real problem with BABIP, but I’m sometimes troubled by the way casual statsters treat the concept of "luck". In statistics luck is, almost by definition, a catch-all term for anything which our model cannot measure with confidence. That doesn’t necessarily mean it’s truly random – indeed, a deterministic view of the universe does not allow that there’s any such thing as pure randomness – only that we haven’t yet figured out how to measure it, or possibly that we have but it’s sufficiently complicated that our largest data samples are still too small to draw any meaningful conclusions from it.

What irks me is when people hide behind the idea of "luck" to assert the primacy of their model. If I’ve got a metric that measures a lot of things but not quite everything, and Nico comes along and says, "I can spot exceptions to your rankings with my own two eyes", I can then say, "No, that’s meaningless; it’s just luck." But all I’m really saying is that it is outside the scope of my model, which tells us absolutely nothing about whether my model is wrong or Nico’s eyes are wrong.

This problem particularly comes up with projective stats. I tend to be far more annoyed by use of projective stats because those who quote them tend to forget that they are merely projections of what will happen, not record of what actually did happen. Predicting what a player is going to do is a lot harder than recording what he did do. You have to make educated assumptions about what past data is relevant and what isn’t. With enough data you can predict pretty well, but you’ll never get it perfect. And then when you do guess wrong, you still don’t know what the significance of your error is. Is it truly just "luck", or is it that your model is flawed and you regressed to the wrong mean? Again, with enough data you can do analyses and get a better idea of that, but you still never really know. Some people aren’t satisfied with such a fuzzy answer so they just pretend the model is always right and anything else is luck, so if Joey Pizzicato was rated a 3.45 prospect but then performs like a 2.34 they say, "oh, he just got unlucky; he’s really a 3.45" because the projected stat said so. But was it really bad luck, or was the projection just dead wrong about Joe? We don’t know.

I’ve spent a lot of time among statisticians. My sister is a professional statistician, and a good friend of mine is a semi-celebrity statistics academic. My own understanding isn’t that deep, but one thing I’ve observed just being around statistics people is that the more one knows statistics the greater one’s appreciation for what they don’t tell you. I think this holds true in the subworld of baseball statistics as well. The guys who are actually generating these numbers, as far as I can tell, know very well exactly what they’re measuring. And the people who spend a lot of time wading through articles on their sites know it a whole lot better than those who don’t. Most of my objections are to the guy at the far end of the spectrum who is eager to have exact numbers to play with, so he looks them up in a table somewhere and then builds a whole world of argument around them without looking too carefully about whether the numbers can really bear his conclusions.

And I would remiss if I didn’t point out that, in my opinion, AN is far better at this sort of thing than most websites (but not better than LL; sorry). Even our yo-yos here aren’t as annoying as the ones I see attaching comments on other sports pages. Our top stats-minded guys here I have a lot of respect for and mostly trust, but I do think that each of them gets carried away occasionally and is too eager to claim more than their numbers can really bear. When I call one of them out on something, it’s not because I doubt their ability nor because I just don’t believe in stats at all; it’s just that I think they deserve to have a skeptical observer questioning their arguments to keep them honest.

Sorry for the long rant. Hopefully it’s something I can apply generally to many other discussions over the months to come.