FanPost

What Causes Statistical Variation? — A Philosophical Debate


Something which has come up, sometimes indirectly and sometimes head-on, in many recent discussions on AN is the following or a variation on the following question:

What causes a player to perform significantly better or significantly worse than his median performance or than his perceived "true talent level?"

Recent manifestations of this question have appeared in the "Mountain out of a Cahill" thread, referring to Esteban Loaiza's career year and whether or not this represented a change in talent or luck, about Cahill in the same thread, referring to his ability to sustain a ridiculously low BABIP, about Pennington and Kouzmanoff in Nico's recent post about their streakiness, and in numerous other game threads and posts throughout the year.

Over the course of a year, a player who ends up with a .400 OBP will have periods with a .500 OBP and periods with a .300 OBP. This is always true, though the sample of each is ever in flux. When flipping a coin, some periods of time will produce strings of 100% heads, other periods will produce 100% tails.

Thus, if the desired result is heads, and a coin is flipped once and lands on heads, the 100% success rate is an unsustainable product of luck. We know that the coin (assuming it is a truly perfectly weighted coin) will eventually regress towards 1/1 heads/tails ratio after enough flips.

But here is the question which I am interested in: What causes EACH result?

We'll start by continuing with the coin analogy, because it's the most simple way of looking at the question. When a coin is flipped, we assume a 50% chance of the coin landing on either side. But as you are well aware, that doesn't mean that the coin lands on its side every time. Rather, it means that every time it is flipped, the coin lands squarely on one side, but the total aggregate of flips will produce a balanced result. So if we know that a singular coin flip will produce a singular result, what are the causes of the result? It cannot be the coin, because as we know the coin is perfectly equal between the two sides. Thus, the result of the coin flip is determined by an indescribable amount of various other factors: wind, angle of toss, speed of toss, material of landing surface, even air resistance. These factors are vast, many are minute, many are uncontrollable, and the ones that are controllable (speed of toss, angle of toss, etc.) are so immeasurable that people do not (rightfully so) expect to be able to control the result by changing them.

So we refer to these factors as "luck."

Luck is a strange concept, and although this is not something that we often consider when discussing luck, it is very much a Western concept. Buddhists, for instance, do not believe in luck as we know it. The oft-misinterpreted concept of "karma" precludes the possibility of luck, saying that all actions are caused by the actions which came before, and thus for someone to describe something as a function of "luck" is merely to be unable to perceive or to quantify the factors which caused the result.

When a hitter plays a game and goes 5-5 with five line drives, that is obviously unsustainable production. Moreover, the hitter's true talent level is undoubtedly not that good—no one hits line drives in 100% of their at bats over the course of a full season. That said, almost ALL hitters do have games where they hit line drives (or at least hit the ball hard) every time they come up. Is that luck?

Now here's the thing. The question I'm asking is a nuanced one, and in some ways seems purely semantical. There is no doubt that the classification of a hitter's "hot streak" of 20 ABs with 10 hits compared to 10 ABs with no hits as "statistical noise" is accurate, just as a "lucky streak" of 10 heads in a row should be called statistical noise. My question here is attempting to go to the next level, not in a practical sense but in a theoretical sense. If we are to assume that "luck" is indeed caused by factors that do not reach the quantification level of the experiment, factors which are too small, too immeasurable, and too intertwined to be able to include within the boundaries of a repeatable and projectable statistical analysis, then one can look at each individual event as it should be looked at—totally separate and categorically unlike any other event.

So then, you will say, no statistic, no percentage (be it OBP, BA, K%, or even the % of heads in a die-flip) CAN EVER be used to predict the result of a single event. And in some ways this makes sense. A die-flip, for instance, because of the 50/50 chances of landing on either side is often called "completely random," meaning that there is no predictive ability. But what about a batter with a .400 OBP? Can't we safely say that, in his next at bat, he is more likely to get out than not?

But here's the thing, and this is crucial to understand when thinking about statistics (particularly in baseball). What on-base percentage represents IS NOT the unchanging likelihood in each individual plate appearance that the hitter will get on base. We already know this, because we know that a .400OBP player is less likely to get on base against Tim Lincecum than against Dana Eveland. What the .400 OBP represents is the ability of the player to SUSTAIN his production throughout an extended variety of events at a cumulative rate of 40% success. On-base percentage would be useless over the course of one PA. Either he gets on or he doesn't, it might as well be in binary. What OBP (and of course this is just a single example) signifies is the aggregate of results over a stretch of time, and the ability of a player to maintain a certain skill level against a variety of competition and whilst enduring a countless number of immeasurable factors. So while Cliff Pennington is hitting nearly .500 for a three week period, we know that he won't be able to sustain such production. But what are the causes of such a streak? Is it weaker pitching? A mechanical fix in his swing? Improved eye-sight? And whatever it is, WE KNOW that it is unsustainable. So does that help us reach a diagnosis? What causes an improved line drive percentage?

Thus, we accept some things as factors (skill of opposing pitcher, amount of foul territory and various park factors, etc.) but not others which we regard as luck.

So in the long run, what am I getting at?

I want to hear your takes on the factors that go into producing A SINGLE RESULT. I want to hear what people think about the understanding of luck and its relationship to scientific quantification, and I want to hear your explanations (preferably quantifiable, because honestly my disbelief in luck is driven by an ultimate belief in quantifiability and that what is classified now as "luck" is simply an opportunity to find the next untapped unexploited statistical advantage) for what could cause someone to be "lucky" when they are.