What Causes Statistical Variation? — A Philosophical Debate
by King Richard on Jul 6, 2010 3:34 PM PDT
Email
Print
Something which has come up, sometimes indirectly and sometimes head-on, in many recent discussions on AN is the following or a variation on the following question:
What causes a player to perform significantly better or significantly worse than his median performance or than his perceived "true talent level?"
Recent manifestations of this question have appeared in the "Mountain out of a Cahill" thread, referring to Esteban Loaiza's career year and whether or not this represented a change in talent or luck, about Cahill in the same thread, referring to his ability to sustain a ridiculously low BABIP, about Pennington and Kouzmanoff in Nico's recent post about their streakiness, and in numerous other game threads and posts throughout the year.
Over the course of a year, a player who ends up with a .400 OBP will have periods with a .500 OBP and periods with a .300 OBP. This is always true, though the sample of each is ever in flux. When flipping a coin, some periods of time will produce strings of 100% heads, other periods will produce 100% tails.
Thus, if the desired result is heads, and a coin is flipped once and lands on heads, the 100% success rate is an unsustainable product of luck. We know that the coin (assuming it is a truly perfectly weighted coin) will eventually regress towards 1/1 heads/tails ratio after enough flips.
But here is the question which I am interested in: What causes EACH result?
We'll start by continuing with the coin analogy, because it's the most simple way of looking at the question. When a coin is flipped, we assume a 50% chance of the coin landing on either side. But as you are well aware, that doesn't mean that the coin lands on its side every time. Rather, it means that every time it is flipped, the coin lands squarely on one side, but the total aggregate of flips will produce a balanced result. So if we know that a singular coin flip will produce a singular result, what are the causes of the result? It cannot be the coin, because as we know the coin is perfectly equal between the two sides. Thus, the result of the coin flip is determined by an indescribable amount of various other factors: wind, angle of toss, speed of toss, material of landing surface, even air resistance. These factors are vast, many are minute, many are uncontrollable, and the ones that are controllable (speed of toss, angle of toss, etc.) are so immeasurable that people do not (rightfully so) expect to be able to control the result by changing them.
So we refer to these factors as "luck."
Luck is a strange concept, and although this is not something that we often consider when discussing luck, it is very much a Western concept. Buddhists, for instance, do not believe in luck as we know it. The oft-misinterpreted concept of "karma" precludes the possibility of luck, saying that all actions are caused by the actions which came before, and thus for someone to describe something as a function of "luck" is merely to be unable to perceive or to quantify the factors which caused the result.
When a hitter plays a game and goes 5-5 with five line drives, that is obviously unsustainable production. Moreover, the hitter's true talent level is undoubtedly not that good—no one hits line drives in 100% of their at bats over the course of a full season. That said, almost ALL hitters do have games where they hit line drives (or at least hit the ball hard) every time they come up. Is that luck?
Now here's the thing. The question I'm asking is a nuanced one, and in some ways seems purely semantical. There is no doubt that the classification of a hitter's "hot streak" of 20 ABs with 10 hits compared to 10 ABs with no hits as "statistical noise" is accurate, just as a "lucky streak" of 10 heads in a row should be called statistical noise. My question here is attempting to go to the next level, not in a practical sense but in a theoretical sense. If we are to assume that "luck" is indeed caused by factors that do not reach the quantification level of the experiment, factors which are too small, too immeasurable, and too intertwined to be able to include within the boundaries of a repeatable and projectable statistical analysis, then one can look at each individual event as it should be looked at—totally separate and categorically unlike any other event.
So then, you will say, no statistic, no percentage (be it OBP, BA, K%, or even the % of heads in a die-flip) CAN EVER be used to predict the result of a single event. And in some ways this makes sense. A die-flip, for instance, because of the 50/50 chances of landing on either side is often called "completely random," meaning that there is no predictive ability. But what about a batter with a .400 OBP? Can't we safely say that, in his next at bat, he is more likely to get out than not?
But here's the thing, and this is crucial to understand when thinking about statistics (particularly in baseball). What on-base percentage represents IS NOT the unchanging likelihood in each individual plate appearance that the hitter will get on base. We already know this, because we know that a .400OBP player is less likely to get on base against Tim Lincecum than against Dana Eveland. What the .400 OBP represents is the ability of the player to SUSTAIN his production throughout an extended variety of events at a cumulative rate of 40% success. On-base percentage would be useless over the course of one PA. Either he gets on or he doesn't, it might as well be in binary. What OBP (and of course this is just a single example) signifies is the aggregate of results over a stretch of time, and the ability of a player to maintain a certain skill level against a variety of competition and whilst enduring a countless number of immeasurable factors. So while Cliff Pennington is hitting nearly .500 for a three week period, we know that he won't be able to sustain such production. But what are the causes of such a streak? Is it weaker pitching? A mechanical fix in his swing? Improved eye-sight? And whatever it is, WE KNOW that it is unsustainable. So does that help us reach a diagnosis? What causes an improved line drive percentage?
Thus, we accept some things as factors (skill of opposing pitcher, amount of foul territory and various park factors, etc.) but not others which we regard as luck.
So in the long run, what am I getting at?
I want to hear your takes on the factors that go into producing A SINGLE RESULT. I want to hear what people think about the understanding of luck and its relationship to scientific quantification, and I want to hear your explanations (preferably quantifiable, because honestly my disbelief in luck is driven by an ultimate belief in quantifiability and that what is classified now as "luck" is simply an opportunity to find the next untapped unexploited statistical advantage) for what could cause someone to be "lucky" when they are.
33 comments
|
5 recs |
Do you like this story?
Comments
I think the key word to why the stats have fluctuation and variance can be summed up in one word
and that word is adjustments.
The game of baseball is such a micro-macro proposition, with forces exerting themselves and then, after sustaining themselves, abating back to a mean once the opposition league-wide has made its adjustments to the exertions. Pennington will hit like Don Mattingly for 3 weeks, accumulating enough video for the other coaching staffs to look at, and the other teams will make their adjustments to whatever is working for Cliff accordingly. Then he will experience the inevitable regression and it will be up to him to make further adjustments to the previous adjustments made against him, which were precipitated by the earlier revolutions of the ongoing Cycle of Adjustment, if you want to call it that.
It’s the same with pitchers… particularly the younger and less-established ones of the variety the A’s seem to be so good at developing. Gio Gonzalez may walk a bunch of guys and work with Curt Young on pounding the zone, or more specifically on throwing his curveball in more riskier situations, and this may provide him with the necessary boost in situational command to curtail his walks and pitch more effectively. Then it is up to the video the other teams accumulate from the successful period, and what it can show about how hitters can swing or take at different times in the count to maximize their leverage against the adjustment Gio has made into what amounts to an adjustment of their own. Once that happens and the walks creep back up, it is up to Gio and his coaches to discern what can be done to put the opposing hitters off the scent, so to speak, so they will be more vulnerable to his pitch selection in more counts and get less favorable results against him. And the dance goes on.
Of course the older players get, and the more experienced they become, the harder it becomes to make adjustments that aren’t going to be transparent to the opposition as adjustments those players have already made in the past, and which have garnered appropriate adjustments-in-response earlier on in the cycle. I think of how Cal Ripken would change hhis batting stance every 2 weeks sometimes, and how as a HoF established player people would sort of make fun of him for that… but to me it was always someone trying to out-adjust the opposition in a pro-active manner.
I think for pitchers it’s maybe a little harder, because they come with at most 4 or 5 different pitches and there’s only so many counts and situations before the statistical reality of what they are doing becomes discernible to opposing coaches and hitters. This may be why we see such thin pitching some seasons, with such significant dropoffs from teams’ 1-2 starters down to their 3-thru-5 guys… because it’s harder to make continual adjustments that can go undecipherable with such a limited perceivable repertoire for them. Maybe this is why there seem sometimes to be less elite pitchers than there are elite hitters, but I could be taking that too far, I’m not sure.
I guess what I’m trying to relate this to in your post is that the statistical fluctuations and differences-in-situational-results seem to me to hang on a skeleton framework of this Cycle of Adjustments, and that whatever the result from situation-to-situation will be depends on at point the players in question are in the Adjustment Cycle that is ongoing between all opposing players at all levels, offensive and defensive, of the game at the micro (single game) and macro (full season/career) aspects of those competitive interactions.
Hello, I'm Vince Cotroneo for the Marmaduke B. Mushmouth School of Public Speaking...
Spot on, doctor...
Plus, a player’s ability to maximize value from the Cycle of Adjustments varies due to multiple factors, like mood, diet, and willingness to adjust in the first place.
Also, the Cycle of Adjustments works at its’ most efficient with a top notch coaching staff and manager in whom the players have confidence.
It also helps A LOT to have one or two guys who can tell when pitchers are tipping pitches, not just to keep our pitchers ahead of the Cycle of Adjustments, but to tip off our hitters on some fat fastballs and some breaking balls to lay off.
New mind-warping, stomach-churning Pilots songs are now online... follow the link if you dare (don't say you weren't warned!) NSFW!!!
by Gaijin_Suketto on Jul 6, 2010 5:29 PM PDT up reply actions
That's no doubt part
And certainly accounts for streaks. But it doesn’t account for the on-off scenarios that also play into the end results.
I think the ultimate problem with quantifying it is that results are probably dependent on a number of things. Maybe the player didn’t drink enough water the day before, didn’t sleep well, had some food disagree with him. Maybe he was mentally worn out from traveling, or bickering with his kids or significant other.
Maybe they just don’t feel it some days. I know that some days I go to the gym, or go running and just nail it. Other days, with similar sleep, it’s a real struggle. Why? I have no idea. Now I’m probably not even close to in-tune with my body as a professional athlete, but you gotta figure they get the same things to some degree.
The problem with trying to predict that sort of stuff though, well it’s not really possible( yet ).
I like the Buddhist view of karma, though more from a physicists standpoint than the macro sense. I still would attribute stuff to “luck” though – basically that which isn’t reasonably controllable by skill. Because while I know that the definition is technically wrong, it’s a reasonable enough approximation for what I’m trying to convey.
Totally agree with this
I like Cindi. A. She never pretends to know more than she does. B. She has unbridled enthusiasm for her "Hotties," and isn't afraid to show it. -IM4Oakgal
Along with EN's spot on analysis, I'd specifically add:
1. “fixing things” (e.g., mechanics in swing or release point in pitch)
2. playing at full health vs. compromised health (e.g., a bad shoulder causes more balls to be popped up instead of squared)
3. level of confidence (which, in either direction, can feed on itself)
4. weather (often relates to health)
I like Cindi. A. She never pretends to know more than she does. B. She has unbridled enthusiasm for her "Hotties," and isn't afraid to show it. -IM4Oakgal
Annie Savoy was correct
And this isn’t philosophical, it’s religious.
Adjustments, I agree, are a huge part of the game and will affect outcomes. And so are batters making educated guesses as to what pitch is coming next and then being able to react in time no matter if they were correctly guessing or not. And, likely, their confidence that they will drive a pitch hard for a hit probably factors in as well.
Having pitched in high school and having been a student of the game for many years, I think that the bulk of the variable lies with the pitcher. When you see a pitcher who is “on”, it’s magical. Being “on” the very few times I actually was, was magical and when it happened. I felt unhittable.
I enjoy a well pitched game and some do not. But there’s one thing that you can say about a pitcher who is in the zone, they’re usually going to get even the very good hitters out. Baseball adages and cliches aren’t always correct but there is one that I firmly believe: great pitching negates great hitting.
I don’t know where else to go with these unfocused and random thoughts of mine so I guess I’ll just end it right he
When I have a good day at work, it's usually because I'm in a good state of mind and
I’m in a situation where my strengths are allowed to come through and my limitations aren’t very important. When I have a bad day, it’s someone else’s fault.
it’s tough to ‘support the laundry’, especially when the teams are losing on a consistent basis. - OldhamA
I almost think of it
In a “first law of thermodynamics” sense: there is some finite pool of luck and how much of it we have changes over time.
Given that, while it’s probably not mathematically accurate, it’s a reasonable assumption to say that subsystems like baseball approximate the universe. In other words, there is finite luck in baseball that’s variably distributed amongst the players at any one time. Further, this also means that baseball luck doesn’t really follow seasons (but for statistical purposes, it’s easier to divide things up by seasons)
So, if we say all that, this means that each player has variable luck throughout his career. It ebbs and flows. This is in contrast to a player’s skill, which follows a bell-ish curve. You have a somewhat steep increase in talent, but it probably drops off slowly.
Why does Pennington, then, go on a hot streak? In a pitcher-vs-batter matchup, it’s both difference in skill-vs-difference in luck. Wherever it’s skewed for that moment in time is who wins. Pennington has the luck right now that is able to overcome skill. This won’t always be the case.
I’ve worked in wet labs, everything from tissue culture to organic synthesis. Sometimes, even in a six-sigma environment, you just suck. Your cells don’t grow because your media has a little too much of something. This reaction won’t complete because you pulled out a dud of the enzyme stock. Sure, you could try and get down to the minutiae of why you sucked, but it’s almost not worth it. You should just try again.
I’m not sure if any of that made sense in retrospect. I need coffee.
"Do I talk to myself? No, I just remind myself of what I'm trying to do. You know, I never answer myself so how can I be talking to myself?" - Rickey
Athletics Nation - WE'RE ALL GONNA MRIIIIIIIIIIIII!!!!! - danmerqury
The Praise
Very interesting write-up, with so many questions/topics touched. If I am not mistaken (or just didn’t get a joke) you were Nico’s student which should in turn make you pretty young. So, I guess everybody using their (young) age as an excuse for not-so-thoughtful writings should take an example here.
As there is so much to be said here, I’ll try to keep my thoughts somewhat organized and will separate it in multiple comments. This way I will also keep improving my paltry comments/FanPosts ratio.
The Linguistics
I am not a linguist at all and definitely not when it comes to English. I do, however, prefer the the word chance to the word luck, when it comes to describing the baseball factors that don’t seem to be controllable.
It might be that my understanding of the given words is wrong, but I always associated chance with a random event with no memory of its own, while luck to me represents a good fortune when one should not be expected.
The random factors in baseball influence the outcome towards both sides of the spectrum – the results are sometimes “good”, sometimes “bad”. Usage of the word luck, and its positive connotation, indicates the influence in positive direction and forces us to use another word, its opposite – “bad luck” – to describe events that lead to the sub-par results, whereas chance alone covers the whole spectrum.
Also, luck more easily triggers association to personal trait. “He is getting lucky” is almost dismissive, as it implies the person somehow received something he did not deserve and almost collaborated in the effort to do so.
I think you might be defining where there is just connotation
Luck and chance are really synonymous. In fact, the first definition for “chance” in the Merriam-Webster dictionary:
1 a : something that happens unpredictably without discernible human intention or observable cause b : the assumed impersonal purposeless determiner of unaccountable happenings : luck c : the fortuitous or incalculable element in existence : contingency
(bold is mine)
www.zekeishungry.com
by thejd44 on Jul 7, 2010 8:53 AM PDT up reply actions
Chance is the fool's name for fate
There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.
The Measurement
For more than a decade now, I have dedicated a big part of my life to development of measurement devices. It should come as no surprise then that I believe that everything is measurable and quantifiable. Some might argue that life is an art that needs no numbers and measurements, I will argue that knowing how, when and what to measure is an art in itself.
We have laser guns and body fat indexes. We have time to the plate and Pitch F/X data. We have hamstring strain degrees and bat speeds. We know some things, but we still do not know most of them.
Once we will have some sort of Batter F/X data we will be able to say more. Bat path to the ball, impact spot, duration and angle, to start with. And that’s only the simplest physics of it. Regardless how it could be measured and whether we want to or not, one can imagine measuring a series of chemical and biological events going on in the athletes themselves. Are they choking? Are there uncontrolled muscle contractions in high leverage situations? Or are they thriving on, say, higher level of adrenaline being pumped in their systems making their reaction time faster and their eyes sharper? Are electrical impulses being sent by the pain receptors “clogging the bases” for the ones trying to trigger a certain muscular response?
All this would only move the frontier, as it has been moved so many times in the past, unveiling a little bit of the gray area that sometimes tends to get black or white in the heated discussions between the “it’s all will” and “it’s all luck” camps. The discussions will never cease, though – the science can only help answer the “hows”, not the “whys”. So, we would still have plenty of material around here to talk about.
by elcroata on Jul 7, 2010 8:04 AM PDT reply actions 3 recs
alright, rec'd
"Life is a horizontal fall" -Jean Cocteau
by King Richard on Jul 7, 2010 10:15 AM PDT up reply actions
‘You are to be in all things regulated and governed,’ said the gentleman, ‘by fact. We hope to have, before long, a board of fact, composed of commissioners of fact, who will force the people to be a people of fact, and of nothing but fact. You must discard the word Fancy altogether. You have nothing to do with it. You are not to have, in any object of use of ornament, what would be a contradiction in fact. You don’t walk upon flowers in fact; you cannot be allowed to walk upon flowers in carpets. You don’t find that foreign birds and butterflies come and perch upon your crockery. You never meet with quadrupeds going up and down walls; you must not have quadrupeds represented upon walls. You must use,’ said the gentleman, ‘for all these purposes, combinations and modifications (in primary colours) of mathematical figures which are susceptible of proof and demonstration. This is the new discovery. This is fact. This is taste.’
sock puppets have never successfully defended castles, except when working with squirrels, which would never happen because squirrels know better than to trust sock puppets. -nm
by Leopold Bloom on Jul 8, 2010 2:49 AM PDT up reply actions
The Excuse
Gotta run to my basketball practice and there is a big soccer game later on. Hope this thread stays up for a while and I can chip in with a couple more.
Thanks for writing, King Richard
This was much more interesting than anything we’ve been getting from Prince John.
"We've come a long way, and I'm not talking about Virginia Slims, either." - Art Howe
What is a streak? A result or a process?
Our inability to come closer towards the elements that define a result (for the lack of proper measurement tools) results in our inability to properly recognize a streak. We analyze the streak of results, that might or might not have been a streak of an improved process. We say, Hunter has had a great week, whereas the data might be skewed by the fact he had a 2-HR day with the jet stream blowing out, blooped a double and was awarded BB on a clear strike three. In that same period Choo might have been squaring on every pitch, but hit four line drives directly at people and had Gutierrez rob him of two further hits by diving plays.
Until we find a better way to measure the process closer to its core (some ways are already there, like LD%), we will be to far down the chain of events to properly evaluate what we call a streak.
I absolutely believe that athletes perform on different levels all the time for many of the reasons that were already mentioned here and some more. However, I am not convinced that in such a complex environment as baseball is, the final, measurable results show the whole truth about the quality of process.
So, as for your question, I believe health, biorhythm, weather, bat size, surface, adjustments, mental cycles, time of day, confidence level… all influence the performance streaks. And that they all play only a part role in result streaks, where they are accompanied by lot of events that might be referred to as baseball noise.
game of inches
I think it’s largely because baseball is a game where minute differences in action create hugely different outcomes. For instance, hitting the ball an inch’s difference on the bat creates a very different ball in play. A half-inch even, especially in terms of “north-south” on the barrel – that’s the difference between squaring a ball up and hitting it over the fence vs. a grounder to short. We as humans just don’t have the fine muscle control to reliably coordinate all our large-muscle systems (arms, legs, torso, to say nothing of your brain, the pitcher, the park, etc.) to perform that precisely on the micro-level, so we get massive fluctuations in outcome/performance even if everything else is relatively equal (health, adjustments, timing, ability to handle stress, etc.). The sweet spot is so small and the large muscle coordinations are so loosely controlled with so much opportunity for error.
Same thing on the pitching side – releasing the ball an inch to the left is going to make a massive difference 60’6" away. Some people are better at it than others (Albert Pujols, Roy Halladay, and so on) so they can control their inches to a greater degree, but nobody has perfect control so there’s always a variability factor. Maybe streaks are periods of time in which for whatever reason we have slightly greater body control. Or maybe not. But I’d guess the inherent variability in the physical action (taking account of the player’s personal skill level) is going to outweigh most any other reasonable component (for instance, if my arm falls off that’s obviously going to be the biggest factor in why I’m not a great hitter anymore). Everybody’s trying to do the same thing every time, we just don’t succeed because the target is so small.
"minute differences in action create hugely different outcomes"
That is almost the definition of a chaotic system, in the literal sense.
There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.
As well as the experience of turning to the left with a bad back.
I like Cindi. A. She never pretends to know more than she does. B. She has unbridled enthusiasm for her "Hotties," and isn't afraid to show it. -IM4Oakgal
What does it mean then that players vary at considerably different rates?
Are some streaky and others consistent? Are players (eg., Sweeney) who more consistently perform within some bounds (+/- X points of some metric) less influenced by external factors, such as weather, grounds, pitcher quality, etc? Maybe it makes sense that Suzuki would be “streaky”, given that he is a catcher who has more to worry about during the game and also is more vulnerable to nicks and scrapes than the average player.
For the sake of face validity, I created a measure of consistency. For each rolling 10-game stretch, does the player’s OPS vary by more than 100 points from his season average OPS? So I look at games 1-10, 2-11, 3-12, and so on. If my eyes don’t deceive me, then players who seem more consistent, such as Sweeney and Barton, will have higher scores on this measure – while players who seem “streaky”, such as Suzuki and Pennington will have low scores.
Sweeney 71%
Ellis 60%
Barton 54%
Cust 41%
Kouz 34%
Davis 33%
Suzuki 24%
Penny 15%
The measure seems to hold up pretty well to my perceptions. I’m a bit surprised by Ellis – I normally consider him to be “streaky”.
It’s amazing that for any 10-game stretch, there is an 85% chance that Pennington will be performing at least 100 points above or below his season average (on this particular metric). Leads me to wonder whether he is just incredibly lucky during his hot streaks and incredibly unlucky when he is cold? Or maybe there is a big upside for Pennington if the team is able to identify and manage (as Sweeney seems to do) the external factors that are associated with his cold streaks. The flip side is that there may be little upside for Sweeney – what we see is what we will get – perpetually.
Next question is whether consistency is consistent – but I don’ t have the time to measure this over different years.
To me the most surprising is that Rajai isn't streakier.
I don’t think Pennington’s streaks especially coincide with luck, just that for whatever reason he has been particularly streaky in 2010. As for “next question,” my guess would be that over different seasons, consistency is not particularly consistent.
I like Cindi. A. She never pretends to know more than she does. B. She has unbridled enthusiasm for her "Hotties," and isn't afraid to show it. -IM4Oakgal
Sitting between Kouz and Suzuki would make Rajai pretty streaky!
Of course, there is nothing inherently good about consistency – Sweeney proves that you can be consistently meh.
It has been observed that certain styles of hitting are streakier than others.
In particular, home run sluggers who walk a lot are supposed to be especially streaky. This could explain Sweeney’s non-streakiness, at least.
I like your rolling-10 number. I’d like to see a program that cranks out something like this for all hitters and then plot it against various hitting stats to look for significant correlations in a large sample.
There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.
Really interesting stuff.
Obviously this is just a quick picture, I’d love to see a more in depth exploration of this sort of stat.
"Life is a horizontal fall" -Jean Cocteau
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Certain photos copyright © 2012 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
