ESPN Roundtable On "Scouts" vs. Stats

Hatfields and McCoys. Montagues and Capulets. Yankees and Red Sox. Creationists and Scientists. Old Scouts and Young Sabermetricians.

Never the twain shall meet.

    Bill James
Sabermetrics was pioneered by current Red Sox advisor Bill James years ago.
But in The Great Debate, Alan Schwarz does his best to bring scouts and sabermetricians together.

Why? Alan is comfortable moving in both worlds, so it's natural that he'd want to bring their inhabitants together. But what he's trying to do can't be done in a meaningful way, because these two worlds cannot occupy the same space, any more than Earth and Mars can occupy the same spot in the solar system.

Everybody gives it the ol' college try, though, at least when you get them in a room together. After all, most scouts and most sabermetricians are nice guys, once you get to know them. And nice guys don't like to shout at each other.

Too, it's politically correct. These days, if you ask a general manager, especially one of the young ones, how he reconciles scouting and statistics, he'll tell you that 1) the key to winning in the 21st century is "blending" the traditional methods of player evaluation with newfangled objective analysis, and 2) these two things are, roughly speaking, equally important.

If you think about the second of those for more than a couple of seconds, you'll realize that of course it's preposterous. If you're scouting a pitcher you've never seen before, you want to know what kind of pitches he throws, and how well he throws them. But would anybody believe those things are precisely as important as each other? In fact, "how well" is a lot more important than "what kind"; anybody would take a great fastball and a good curveball over five sub-par pitches. If scoring runs and preventing runs aren't equally important (and they're not), then nothing is.

So let's dispense with the notion that seat-of-the-pants opinions and objective analysis are exactly as important as one another, because they're not. The odds are a million-to-one against. Mind you, I'm not saying which is more important (though of course I've got an opinion on the matter); I'm saying that it's a cop-out to suggest they're equivalent.

And here's why the old-line scouts and the objective analysts will never get along, really: They can't get along, in the end, because their belief systems are diametrically opposed. Let's assume, for a moment, that you spent a year of your life working, full-time, on a study that compared high-school pitchers to college pitchers. Let's assume, too, that your methods are impeccable, the conclusion is that high-school pitchers aren't worth the trouble of scouting (let alone paying) them, and that you're able to explain all of this in a way that any reasonably intelligent person could understand.

There aren't a dozen old scouts in America that would believe you. Actually, I'm not sure you could find even one.

Last month, there was a great piece in Slate by Henry Blodget, in which he enumerated all the ways that investors fool themselves. All of it's instructive, but I found this highlight particularly applicable to baseball:

    Conservatism Bias and Confirmator Bias: Once we form opinions, we tend to overvalue information that reinforces them and undervalue information that undermines them (conservatism bias). We even tend to seek out supporting information (confirmatory bias). Thus, we irrationally cling to incorrect conclusions, and, to paraphrase Simon and Garfunkel, hear what we want to hear and disregard the rest.

Every time you tell an old baseball guy that high-school pitchers are relatively poor bets, he'll toss off a bunch of names, usually beginning with Greg Maddux and ending with Zack Greinke. Of course, for every Maddux and Greinke, there are any number of Brien Taylors and Kiki Joneses. But that's not fair, is it? I'm doing the same thing, just tossing names out there. And I promise you, if you study the issue with great rigor and discover it makes more sense to draft high-school pitchers than college pitchers, I can find a few hundred sabermetricians, professional and amateur, who will pore over your study. And if you're right, you'll change a lot of minds.

But for the older scouts (and I'm speaking generally here), of course there are exceptions. Scouts simply won't believe there's nothing special about high-school pitchers, because if they believe that -- along with a host of other chestnuts -- then they have to reevaluate their own worth in the world, since so much of their value resides in their supposed ability to predict what a 17-year-old pitcher will be doing in six or eight years. And reevaluating your worth -- in this case, downward -- isn't fun for anybody. So scouts cling to their beliefs; they overvalue the information that reinforces them, and undervalue the information that undermines them.

Yes, sabermetricians are subject to the same psychological foibles ... except sabermetricians have a balancing crutch: the data. In the world of investing, John Henry and Warren Buffett have the same sorts of brains that you and I have. Oh sure, they're a bit smarter than the rest of us. But what really separates them is that they believe the data, and avoid letting their "innate tendencies" get in the way of making the smart plays.

Look, I'm not saying that old scouts and young sabermetricians can't be friends, or that they can't respect each other. I know people who believe that men and dinosaurs once roamed the earth together, a few thousand years ago (like in "The Flinstones," but without the cool cars). That's patently false, and I don't feel obligated to respect any belief that's patently false (even while acknowledging that some things I consider patently false will someday be proved true). I still respect the people, though.

But people in baseball say they respect not only the other people, but also the other beliefs. In "The Great Debate," Angels director of scouting Eddie Bane says, "We believe there's room for statistical analysis. But we also certainly believe that there's plenty of room for scouting. When Paul Weaver's been with the Astros for 25 years, and he loses his job, and he's an outstanding scout ... that's when we get concerned. If there's $50,000 in the budget, and you're going to hire a scout or a stat guy, one guy's going to be out in the street. And the guy who was out in the street was Paul Weaver."

According to the 2004 Houston Astros Media Guide, last year the Astros employed 38 full-time scouts in the U.S. alone, along with a number of scouts in Latin America (some of them less than full-time, probably) and the usual squad of coordinators, directors, and assistant coordinators and directors. So I'm not sure who's saying there's not plenty of room for scouting, but it certainly isn't the Houston Astros (who, by the way, don't list anybody in their media guide who's specifically devoted to statistical analysis).

See, the problem with scouting -- and so many other walks of life -- is that once you've had the job for a few years, it's sort of assumed that you're owed that job. Nobody really knows who the good scouts are, so instead it's assumed that you're a good scout if you've been at it for a while. Where's the evidence that Paul Weaver is "an outstanding scout"? Why, he's been with the Astros for 25 years! As if there aren't lousy bus drivers, college professors, and Certified Public Accountants who've been at their jobs for 25 years. Tenure is not talent.

    Paul Depodesta
Paul DePodesta fully embraces the power of sabermetrics.
For all I know, Weaver really is an outstanding scout. And if he is, there were probably 29 other teams lined up to hire him the moment the Astros so foolishly cut him loose. Wouldn't a truly outstanding scout be worth his weight in silver? That's the way the world is supposed to work, right?

I've digressed. The point is that everything's different now. It's not that scouts are an endangered species. Sabermetricians crave information, and some of that information can be gathered only by a scout with either a great deal of intelligence or training (or, ideally, both). One misconception about sabermetrics is that it's about statistics, and only statistics.

It's not. You give any good sabermetrician two pitchers with similar statistics against similar competition, and still he'll want to know more. What's each pitcher built like? How hard does he work on his craft? How hard does he throw? Those are just three questions, and a curious mind could easily come up with a dozen others. And it's the scouts who will answer most of those questions.

But everything's different now. The rules have changed, and there's no use trying to put the clowns back in the little car. A couple of years ago, I asked Paul DePodesta to name some of the books that guided his thinking, and the first that came to his mind was Thomas J. Kuhn's "The Structure of Scientific Revolutions." And it's right there, on page 150 ...

    Both [groups of scientists] are looking at the world, and what they look at has not changed. But in some areas they see different things, and they see them in different relations one to the other. That is why a law that cannot even be demonstrated to one group of scientists may occasionally seem intuitively obvious to another. Equally, it is why, before they can hope to communicate fully, one group or the other must experience the conversion that we have been calling the paradigm shift. Just because it is a transition between incommensurables, the transition between competing paradigms cannot be made a step at a time, forced by logic and neutral experience. Like the gestalt switch, it must occur all at once (though not necessarily in an instant) or not at all.

Scouts are, in a way, scientists. But for the most part they're poor scientists, and the ones who don't "experience the conversion" will become obsolete. Or if not obsolete, practically unemployable above a certain level in the organization. Do you want to be the one who tells them? Or should I.

Funny that DePo referenced Kuhn - as I started reading Neyer's take, that was the first thing to pop in my mind - "incommensurable".