Maybe this is a stupid stats question

So to sum it up, you’re asking whether it would be good practice to focus on a hitter’s performance against elite pitching (you said top 10%, I believe) for talent evaluation/projections, and visa versa?

It would be interesting to see those kinds of splits, similar to low/med/high leverage situations, but also seems like a hassle to assemble. Plus, it seems like defining that elite hitting/pitching would be somewhat arbitrary. Also, looking at head-to-head stats won’t tell the whole story. Maybe this player absolutely crushes a certain pitch, no matter how elite the pitcher is, while he can’t even hit another pitch thrown by a horrible pitcher?

For minor leaguers, would you think that a player that outperforms his overall stats against the better pitchers be a better prospect than a batter who outperforms his overall stats against the weaker pitchers?

I think that would be rare to find a player that does better against elite pitching than pitching as a whole. If so, I would think it could be attributed to luck and/or small sample size. Otherwise, that player would not be necessary “better” if he can not deal with league-average or poor pitchers, which would account for much more of his at-bats. I see where you’re coming from though. Say there is indeed a player that out-plays his stats against elite pitchers. I wouldn’t say so much that he was “better” than having a higher ceiling. Does that make sense? Kind of like he has a good chance of being a better player, but only if he can tighten up the holes in his game.

Even the ‘best’ stats for hitting and pitching take many at-bats to settle down into an accurate measure of a pitcher or hitter’s ‘true talent level’ (actually, I would argue no one can ever know someone’s true talent level, only their observed performance level…).

Given that, when you start slicing up the pie so that you are looking at only the top 10% of pitchers at a given level, there are too few at-bats to provide reliable data vs. noise.

All kinds of splits (such as home vs. away performance, batter vs. specific pitcher) are almost instantly available these days, but often they are way too small a sample size to say much meaningfully.

I’ll admit, when I’m watching a game and they show me that so-and-so is 0-16 against my pitcher, I’m much more confident than if so-and-joe is 9-16 against him, but statisticians would say neither is anywhere near a big enough sample size to make predictions…

Is to first break up a hitter’s season by pitch type. You can then rank each pitcher by their individual pitch effectiveness and perhaps then as danh (really, another dan on this site talking stats?) said divide those individual pitches into a hierarchy (top 50% and bottom 50%) and see how a hitter does against each half.

The devil in all of this as others have pointed out is sample size. Every time you subdivide the data set you’re losing certainty in the results of the analysis.

Specifically talking about the minor leagues, the first thing that would need to happen is an assessment of the variability of minor league pitching. I’ve no idea what that breakdown looks like. But I know you have career AAAA guys, some young studs, some rehabbing MLBers, all sorts of pitchers are floating around the league. Determining the “top” AAA talent is a chore unto itself.