Athletics Nation: An SB Nation Community

Navigation: Jump to content areas:



Sports blogs for fans, by fans.
New Blog: Backing the Pack for NC State Fans!


The Lowdown on tRA

Hi AN. The more cosmopolitan of you may recognise me as one of the people who writes  for Lookout Landing, and the statistically inclined may be aware that I developed an advanced pitching statistic that's gaining traction across SBN (including the occaisional mention here) called tRA. I'm hoping that I can explain the motivation and methodology behind this stat to all comers in this diary*, and answer any questions folks here have about its use.

Please forgive the British English.

Motivation

As measures of pitcher skill wins, ERA, and WHIP don't cut it. The holy grail of pitching analysis is to determine pitcher skill independently of his team, and anything involving hits is not going to do that. A's fans are gifted with one of the top defenders in the game in the form of Mark Ellis, and I imagine that there won't be much argument from anyone that he makes your pitchers better with his glove.

Except there's a problem. A pitcher should posses the same ability whether he's pitching in front of the best defensive unit of all time or on a team of eight Jack Custs. The defence helps after the pitcher has done his thing. In order to get a handle on pitcher skill, we must somehow look at what happens before his teammates get involved.

Method

We can divorce the defence from our pitcher by looking at pure pitcher outcomes and batted ball profiles.

The former consist of strikeouts, walks, hit by pitch, and home runs. Barring some robbed homers and the possibility of Jose Canseco being involved, fielders have nothing to do with these outcomes. Batted balls mean the distribution of ground balls, line drives, fly balls, and infield popups. These are the plays that the defence can turn into outs.

I imagine that the vast majority of AN'ers are at least passably familiar with FIP, which is a statistic that weights the pure pitching outcomes against runs scored, and puts the result on an ERA scale. While an extremely useful stat and an upgrade on ERA, FIP fails to take into account the pitcher's ability to exert at least partial control over batted ball profiles.

tRA does. By determining the average runs scored and outs made on each of the defence-independent outcomes in each season and league, we can credit a pitcher with runs and innings pitched simply by looking at statistics that they have control over. The following statistics are accounted for in the tRA model:

Strikeouts (K)
Walks (BB)
Hit By Pitch (HB)
Ground Balls (GB)
Bunts (BU)
Line Drives (LD)
Outfield Flies (OFB)
Infield Flies (IFB)
Home Runs (HR)

The actual act of deriving the average outs/runs for each of these over the course of a season is fairly involved, so I won't go into detail here.

Once you know expected outs and expected runs, it's easy to calculate tRA:

xRuns/xOuts*27. There. Done. We now have a metric on the R/9 (NB: not ERA) that is completely defence independent. Is that enough? Not quite. Parks need to be taken into account. This is actually a relatively easy adjustment, made possible by a tonne of work from the folks at The Hardball Times. So now we've derived a park neutral, defence neutral pitching statistic. Hooray.

I hope that was easy enough to follow. If you have any questions, I'd love to hear from you.

*It's not a fanpost dammit.

8 recs | Comment 50 comments

Story-email Email Printer Print

Comments

Display:

Yep

Everything’s available on Statcorner, including regressed tRA, pitching runs above average, etc.

by Graham on Dec 15, 2008 9:27 PM PST to parent up reply reply actions actions   0 recs

404'd

damn you SB-Nation and your crappy links!

you probably meant this.

facepalm.jpg

by Zonis on Dec 15, 2008 9:30 PM PST to parent up reply reply actions actions   0 recs

Is too a fanpost.

And an interesting one at that. I am glad for the explanation. I have seen this stat mentioned fairly often and it didn’t make sense. After reading this I kind of get it. TY for posting.

by IM4Oakgal on Dec 15, 2008 9:24 PM PST reply reply actions actions   0 recs

tRA is certainly an awesome advancement. What about Park Effects?

It's not the results, it's how you look going about those results -- Tim McCarver

by WaddellCanseco on Dec 15, 2008 9:25 PM PST reply reply actions actions   0 recs

We use THT's park factors for the Major Leagues

Link (spreadsheet).

We’re trying to reverse engineer their work for the minor leagues, but going is slow.

by Graham on Dec 15, 2008 9:29 PM PST to parent up reply reply actions actions   0 recs

You're one impressive dude!

It's not the results, it's how you look going about those results -- Tim McCarver

by WaddellCanseco on Dec 15, 2008 9:34 PM PST to parent up reply reply actions actions   0 recs

Hmmm....lager, bitter, stout...

It's not the results, it's how you look going about those results -- Tim McCarver

by WaddellCanseco on Dec 15, 2008 10:07 PM PST to parent up reply reply actions actions   0 recs

Yummy!

It's not the results, it's how you look going about those results -- Tim McCarver

by WaddellCanseco on Dec 15, 2008 10:13 PM PST to parent up reply reply actions actions   0 recs

Any inclination to/thoughts on incorporating pitch-by-pitch data

like swinging strikes, or any of THT’s plate discipline stats?

No, I have no idea how one would go about doing this.

Your 2008 Athletics: It's Nothing Personal.

by PaulThomas on Dec 15, 2008 10:23 PM PST reply reply actions actions   0 recs

Yeah, we're exploring regression of Ks and BBs using pitch by pitch data

Matthew’s done a lot of work on this independently of tRA, so it should be pretty straightforward to incorporate into tRA*. It’s just down on the list.

by Graham on Dec 16, 2008 7:43 AM PST to parent up reply reply actions actions   0 recs

You promise me British English

(better yet, being British, you apologize for it) and then you drop a “tonne” on me !?

by green star oakland on Dec 16, 2008 12:46 AM PST reply reply actions actions   0 recs

Wait a miunte...

Jack Cust is a bad fielder?

I see a deranged rabbit, on fire, cowering away from a vagina. I await the results of the Rorschaschererer. -Nico

by Leopold Bloom on Dec 16, 2008 6:24 AM PST reply reply actions actions   0 recs

Clearly these statistics are flawed

Webmaster of Driveline Mechanics
http://www.drivelinemechanics.com - An Unconventional Look at Scouting

by Kyle Boddy on Dec 16, 2008 6:25 AM PST to parent up reply reply actions actions   0 recs

I got on board StatCorner when the first verion came out

and I think it’s a wonderful website! I pretty much stick to that and FanGraphs nowadays.

Anyway, i think it’s interesting that tRA* shows that Gio Gonzalez wasn’t all that great in Sac last year, and his performance in the majors, using that metric, was pretty similar.

Also, Ziegler’s tRA* is 3.95 and Duke’s is 4.51. Of course, I’m just pulling the one’s with the most luck last year, but they should be regressing significantly this year, which has to affect our chances somewhat.

Cool stuff, that tRA.

"I'm on hold for now"- Bobby Crosby

by DyeLongJustice on Dec 16, 2008 7:06 AM PST reply reply actions actions   0 recs

I'd be very hesitant to apply this to minor leaguers

Perhaps I’m reading it wrong, but it seems like they’re using “studies” (or just Y-T-Y correlations) to determine how much pitcher skill is involved in controlling each event. But these “studies” are only major league pitchers, so there’s very little reason to think they can be applied to minor league pitchers.

by Danny on Dec 16, 2008 8:11 AM PST to parent up reply reply actions actions   0 recs

It's gotta be better than nothing.

It's not the results, it's how you look going about those results -- Tim McCarver

by WaddellCanseco on Dec 16, 2008 8:13 AM PST to parent up reply reply actions actions   0 recs

We run them on each league each year.

This includes every minor league down to A-ball.

by Graham on Dec 16, 2008 8:15 AM PST to parent up reply reply actions actions   0 recs

Can you explain how you do that?

Is it just players who play in the same league for years X and X+1? Is it the correlation to major league performance?

My concern is that—especially for pitchers—the measures we use to differentiate skill from chance/defense/park are not necessarily the same we should use for minor leaguers.

by Danny on Dec 16, 2008 9:17 AM PST to parent up reply reply actions actions   0 recs

why not use the same measures?

by my count, baseball in the minor leagues follows the same rule/structure as baseball in the major leagues. What’s so different about a hitter/batter relationship in the minors vs. the majors? Will you need to use different park measures, but it’s based on the same principles.

"I'm on hold for now"- Bobby Crosby

by DyeLongJustice on Dec 16, 2008 12:58 PM PST to parent up reply reply actions actions   0 recs

Think of a starker example

General tools like DIPS or FIP work reasonably well at the MLB level, but they wouldn’t work at all in little league.

Basically, when we regress HR/FB nearly all the way back to the mean, we’re not saying that pitchers have no control over HR/FB. Rather, we’re saying there’s little discernible difference in the ability of Major League pitchers to prevent HR/FB. If I got called up to the majors, a lot more than 11% of the flyballs hit against me would become HRs. Regressions like this work in MLB because MLB pitchers are part of a selectively sampled group of pitchers who are good enough to pitch in the majors.

by Danny on Dec 16, 2008 3:12 PM PST to parent up reply reply actions actions   0 recs

"pitchers who are good enough to pitch in the majors"

So did you delete Barry Zito from the data set?

it is not possible to strategize while the ball is coming towards you

by eastcoasta'sfan on Dec 18, 2008 8:37 AM PST to parent up reply reply actions actions   0 recs

I misread what you were asking initially

tRA works for the minor leaguers. This is because line drives and homers, which are the key indicators of how hard a pitcher is being hit, are accounted for. Assuming GB and FB have similar out/run values across the league is also reasonable because the batted ball types are essentially bounded by the possibility of a home run. As to the principle being accurate: there should be no difference in methodology between a minor league baseball analysis, a little league analysis, or MLB. The rules are the same. tRA’s weakness w/r/t minor leagues is that we can’t park adjust it properly just yet.

tRA* is based on MLB correlations, and does not work very well on minor leaguers because the regression values will be different and there’s no sensible way of accounting for that. The sample sizes associated with minor league pitching are also small enough that the regression algorithm will often simply throw its hands up and decide that everyone is average. So yeah, it’s less valid that at the major league level.

by Graham on Dec 16, 2008 6:02 PM PST to parent up reply reply actions actions   0 recs

Yeah, I agree with all of that

I don’t think tRA is necessarily a good tool to measure the talent of minor leaguers. If a pitcher in the Midwest League induces a ton of infield flies, for example, we really have no idea whether that’s a talent that will express itself at the major league level or not. As you said, regressing it doesn’t help, because we don’t know what regression to use.

tRA is basically an update of Bill James’ old cERA—which has value, but is a very rough tool to be judging talent with.

by Danny on Dec 17, 2008 11:04 AM PST to parent up reply reply actions actions   0 recs

Except that cERA didn't use batted ball types.

That’s the major step forward in my view.

It's not the results, it's how you look going about those results -- Tim McCarver

by WaddellCanseco on Dec 17, 2008 1:50 PM PST to parent up reply reply actions actions   0 recs

"I don’t think tRA is necessarily a good tool to measure the talent of minor leaguers"

But it does measure, very accurately, how many runs they’d have given up in front of an average defence.

by Graham on Dec 17, 2008 2:02 PM PST to parent up reply reply actions actions   0 recs

No, it doesn't

It makes no differentiation between a GB single through the right side that no 1B/2B could have gotten to and a GB single to the right side that any decent 2B could get.

It makes no differentiation between pitchers inducing especially easily-fieldable BIP and a pitcher who plays in front of a great defense.

It also assumes there’s no difference in ability for pitchers to pitch with runners on or w/RISP beyond their overall abilities—which is a whole other subject.

by Danny on Dec 17, 2008 2:59 PM PST to parent up reply reply actions actions   0 recs

Um

“It makes no differentiation between pitchers inducing especially easily-fieldable BIP and a pitcher who plays in front of a great defense.”

Of course it does. A pitcher who allows especially easily fieldable BIP will give up far less line drives and home runs than his compatriots. These two BIP types almost by definition describe how hard a ball is hit.

by Graham on Dec 17, 2008 4:19 PM PST to parent up reply reply actions actions   0 recs

Um

First, no, it doesn’t make any differentiation between a hard hit grounder and an average hit grounder.

Second, it makes no differentiation in the direction a ball is hit. Your stat makes no differentiation between routine GBs hit right at the SS that I could field and GBs that are hit down the line that Pujols/Beltre couldn’t field.

Not all GBs, LDs, and FBs are equal to each other. I would guess this is especially true in the minors.

Your system “does measure, very accurately, how many runs they’d have given up in front of an average defence” only if you assume that all BIP of a batted ball type are identical and that all events are randomly distributed.

by Danny on Dec 17, 2008 4:44 PM PST to parent up reply reply actions actions   0 recs

You assume that batted balls are discrete buckets

They are not.

1) The harder you hit a ball, the more likely it is to be a line drive/home run. Ergo, there is no major problem with assuming that ground balls are more or less equally hit.

2) Pitchers as the major league level have never shown any ability to determine spray patterns. To determine if such a split exists at the minor league level, we merely must look at whether there’s a major difference in RV/GB etc for players with high RA vs. low. There is not.

The assumptions are valid. Yes, the metric could be improved by incorporating detailed hit vectors, but you seem to believe that the difference would be major, when in actually it wouldn’t.

by Graham on Dec 17, 2008 5:04 PM PST to parent up reply reply actions actions   0 recs

"More likely" is a pretty loose term
1) The harder you hit a ball, the more likely it is to be a line drive/home run. Ergo, there is no major problem with assuming that ground balls are more or less equally hit.

It’s quite obvious that some GBs are much tougher to field than other GBs.

To determine if such a split exists at the minor league level, we merely must look at whether there’s a major difference in RV/GB etc for players with high RA vs. low. There is not.

Do you have a link to this? Thanks.

by Danny on Dec 18, 2008 7:54 AM PST to parent up reply reply actions actions   0 recs

Are you arguing that the the ability

To hit ground balls harder than your peers doesn’t translate (or necessarily translate) towards hitting line drives and ‘hard’ fly balls?

Do you have examples of players who had many ground ball base hits without a high line drive% or home run total over a substantial amount of PA?

by Josh Deletchi on Dec 18, 2008 8:56 AM PST to parent up reply reply actions actions   0 recs

No, I'm not arguing that

First, we’re talking about pitchers.

Second, we’re talking about minor leaguers.

I’m saying that we don’t know how well GB, FB, and LD% for minor leaguers translate in MLB talent in terms of run prevention.

by Danny on Dec 18, 2008 9:14 AM PST to parent up reply reply actions actions   0 recs

How much better than FIP or dERA is this?

Do you have any results along the lines of “Year N tRA correlates better with year N+1 FIP (or ERA) than year N FIP does”?
And o you know how much things like “run value of a fly ball” might vary from pitcher to pitcher, even over large sample sizes? (I use that example because I remember reading somewhere that ground ball pitchers have a higher HR/FB percentage than fly ball pitchers do.)

Thanks for tomorrow 'cause I've had enough

by andeux on Dec 16, 2008 10:14 AM PST reply reply actions actions   0 recs

I have a result from initial testing* of MLB correlation of tRA*(n) to tRA(n+1)

It was around 0.7, where ERA is at 0.29 and FIP is at 0.48. We’re looking to get a larger sample soon.

*From a few years ago when the entire thing was a spreadsheet where all of the data were manually entered.

by Graham on Dec 16, 2008 6:04 PM PST to parent up reply reply actions actions   0 recs

That is impressive

But it’s also only part of the story. It indicates that what your stat is measuring is mostly skill (i.e. repeatable), but it doesn’t tell us how that skill actually relates to preventing runs. As an extreme example, I could invent a stat, call it IRA (Idiotic Runs Against) that assigns run values to a pitcher’s height, weight, shoe size, hair color, and how high he wears his socks. This would obviously have a strong year-to-year correlation with itself, but it would equally obviously not be a good measure of how good a pitcher is at preventing runs.

Since the advent of DIPS people have been eager to use batted ball data to further improve our ability to distinguish “skill” (those aspects of performance that are repeatable) from “luck” (which in this context comprises both random variation and other factors outside the one players control, like park effects and his teammates’ fielding). PrOPS, for example, is based on the premise that batted ball types for an individual batter are skill, and what happens to those batted types after they leave the bat is luck, and (I believe) regresses the former not at all, and the latter 100 percent (e.g. assumes all ground balls are created equal) to get a predicted “luck-neutral” OPS. The problem with this approach is that it’s simply wrong. There’s no sound a priori reason to assume that what percentage of a player’s ground balls go for hits is out of his control. Instead, the right thing to do is determine empirically how much to regress each component of performance(both the batted ball types, and what happens to those batted ball types).

It looks to me like tRA avoids the worst of these problems – I see by the writeup that you do some regression on the components involved, which is certainly the right thing to do. But in order to know that it is an improvement on FIP-type stats, it seems to me we would need to know that the variation among pitchers in the “skill” of getting different outcomes for each particular type of batted ball, is less than the variation of that skill for all batted ball types taken together. Otherwise, separating out the different batted ball types doesn’t necessarily gain anything in predictive value.

Just to be clear, I don’t want it to sound like I’m denigrating this stat. It sounds like you’ve done quite a bit of good work in developing it. But I think there’s still more work to be done in order to show that it’s an improvement on what we already have.

Thanks for tomorrow 'cause I've had enough

by andeux on Dec 17, 2008 2:49 PM PST to parent up reply reply actions actions   0 recs

There's some interesting stuff sort of relevant to that here

http://www.lookoutlanding.com/2008/3/21/182633/721

I go to statcorner not necessarily for tra, but because it has the best, and often only, presentation of the various components.

The A's colors are green and gold.

by mikeA on Dec 17, 2008 3:05 PM PST to parent up reply reply actions actions   0 recs

Thanks

Actually it looks like his followup post:
http://www.lookoutlanding.com/2008/3/22/192722/701
directly addresses some of my questions.

Thanks for tomorrow 'cause I've had enough

by andeux on Dec 17, 2008 3:24 PM PST to parent up reply reply actions actions   0 recs

Comments For This Post Are Closed


User Tools

Welcome to the SB Nation blog about Oakland Athletics.

Community Guidelines ANcillary Terms
Start posting about the Athletics »

Join SB Nation and dive into communities focused on all your favorite teams.

FanPosts

Community blog posts and discussion.

Recommended FanPosts

Small
2010 Oakland Athletics: Internal and FA Options
Small
Possible Trade Partners: Giants
Tsgirlbass_th_small
DLD 7/1/09: Less about baseball, more about baseball movies
Small
Possible Trade Partners: St. Louis Cardinals
Tsgirlbass_th_small
DLD 6/30/09: Pardon me if I'm sentimental

Recent FanPosts

Small
next A's philosophy: High $ International FA signings and High draft picks- pitchers only
Small
A's demands for Holiday
142564582_9b68f505fa_small
GOG 2009 #27
Small
Minor Athletics on vacation -- Minor League Dump
Ziegler160px_small
Quick Texas League All-Star Game Recap
Small
Does Sabean still like gritty veterans?
Small
A'S/Red Sox 2001 classic game on now on A's CSN channel 698 on direct tv
Ph3spec_small
doctorK invades PNC Park

Post_icon New FanPost All FanPosts Carrot-mini

SPONSORS


Managers

Baseball_small baseballgirl

Poochini-butt_in_box_2_small Nico

As_kings_cal_small louismg

08-_the_author_small 67MARQUEZ

Martin_brodeur_300_small Blez

Ziegler160px_small Flashfire

Editors

Countdown_small Taj Adib

527918550406_0_bg_small notsellingjeans

Authors

P1010266b_small devo

Super_grover_small grover

The_baby_small salb918

Official Partner of Yahoo! Sports