BPs Nate Silver to appear on Colbert Report tonight (10/6)

by MrIncognito on Oct 7, 2008 5:28 PM PDT Comment 19 comments

So how does the baseball analyst that created PECOTA end up on the Colbert Report with less than 30 days until a presidential election?

Turns out that Nate is a big time political junkie, too. He's created a web site that uses methods similar to the those used by PECOTA to forcast baseball statistics to predict the result of the presidential election. Also see recent appearances on Olberman or this interview with Dan Rather.

0 recs | Comment 19 comments

Story-email Email Printer Print

Comments

I thought this post was about

Guess I’ll read every word next time.

I'm here to talk about the past.

by 67MARQUEZ on Oct 7, 2008 7:11 PM PDT reply actions 0 recs

Nate Tha' Great

Not a bad poker player either

Can't get enough of the Oakland A's? Visit Oaktown Awesomer's

by iamawesomer on Oct 7, 2008 9:45 PM PDT reply actions 0 recs

Unfortunately

Nate is something of a super liberal.

by VORP is too nerdy on Oct 7, 2008 9:55 PM PDT reply actions 0 recs

Let me clarify:

I got my bachelor’s degree in Political Science. The problem with doing any kind of research in the political field is that you have the danger of introducing your own biases into the objective processes. Additionally, if you’re public about your views (as Nate is) you run the risk of people not taking your research or conclusions seriously because of accusations of bias.

That said, I don’t know how well the PECOTA system translates to making political predictions. The fundamental problem with polling data is that it’s used to suppose how voters will actually behave. The difference between baseball stats is that the data is usually based (at least primarily) on past performance. Political polling data is not always that reliable because there are subtle differences between responses to polls and actual voter behavior.

For example, Republicans traditionally have slightly better turnout than Democrats. Thus, polls involving a strong Republican candidate versus a strong Democratic candidate tend to skew slightly toward the Democratic candidate. The reason is that the Republicans who respond to polls are more likely to actually go out and vote Republican than the Democrats are.

by VORP is too nerdy on Oct 7, 2008 10:05 PM PDT up reply actions 0 recs

But any analyst worth his salts knows that (eg, anyone not employed by one of the networks or the major cable news channels) ....

And Nate Silver is an excellent analyst. He runs the numbers. Numbers are numbers. Numbers aren’t biased — and, while I haven’t looked at his forecast methodology for the elections, I assume, based on his baseball work, that he has done his due diligence and back checked it against every election for as long as comparable polling data has been available. (granted, there aren’t that many, so there are SSS issues) You may think the editorializing that goes along with his numbers are slanted — but, SSS issues aside, I don’t think there’s any reason to question the neutrality of the numbers themselves.

"It's for your own good. Big strong Devo knows whats best for Poppy" -- Mossback

by devo on Oct 7, 2008 10:17 PM PDT up reply actions 0 recs

Numbers aren't just numbers.

It’s how you collect the numbers and how you interpret the numbers that counts. Part of the problem in this regard is that politics is a pretty delicate subject; it has far fewer and far less severe consequences compared to baseball. Polling data is hardly perfect, for example. Nate’s been pretty hush-hush about his methodology. That, and given that he’s worked extensively (and I think exclusively) with leftist groups in the past, it wouldn’t surprise me if he was accused of having an agenda.

The reason I mention this in the first place is that I’ve seen some of the data collected by Nate on this subject, and it looks considerably different from what is typical of a U.S. Presidential Election. Time will tell as to whether this election is considerably different from recent presidential elections, but I have a feeling Nate might be over/underestimating voter turnout.

by VORP is too nerdy on Oct 7, 2008 10:41 PM PDT up reply actions 0 recs

He uses all of the numbers, so there isn't an issue with how he collects them.

He weights polls based on how recent they are and how large of sample sizes they used.

I’m sure he has a political agenda. Why in the hell would anyone devote a huge chunk of their life to working on this stuff if they didn’t really care?

While politics may be more important than baseball, his analysis has no direct effect on anything.

Of course his analysis looks completely different from what is typical — no one has ever done this kind of analysis publicly before. The same could have been said about his baseball analysis eight years ago.

There’s a fairly extensive explanation of his methodology in the faq and he specifically discloses that he makes no assumptions about turnout — relying instead on the pollsters’ assumptions — which, almost invariably lean strongly towards assuming a very normal turnout.

What if any assumptions do you make about turnout? I don’t make any assumptions about turnout. The pollsters make various sorts of assumptions about turnout, and I rely on the pollsters. The only exception is in calculating the popular vote percentage shares for each candidate. For this purpose, I assume that the same proportion of the electorate will turn out in each state as turned out in 2004. However, the turnout figures are adjusted based on changes in the eligible voter population in each state since 2004.

"It's for your own good. Big strong Devo knows whats best for Poppy" -- Mossback

by devo on Oct 7, 2008 11:01 PM PDT up reply actions 0 recs

You don't need to have an agenda to have an interest in politics.

I have never had a political agenda, and I’ve always been interested in politics. Some people are just fundamentally interested in the subject.

The problem is that political analysis has been around for a long, long time. Sabermetrics is a fairly “recent” concept when it comes to analyzing baseball performance, but political analysis has been around much, much longer and has a far more sophisticated history. His methods can’t possibly be as revolutionary to politics as they were to baseball. Also, he doesn’t go into the same type of detail regarding his methodology as you’d see in a typical peer-reviewed study.

Pollsters’ data tends to have certain biases themselves, as you have pollsters from different sides of the aisle. My point here is that a lot of that data Nate is collecting is arguably from biased, subjective sources. It’s impossible to have objective, reliable data under those types of conditions. This is why I’ve never been a big fan of polling when it comes to predicting electoral results.

But this is a blog we’re talking about. Nate can write whatever he wants on his personal blog. But I get the feeling that the some elements of the media present it as legitimate scientific research, which is certainly is not.

by VORP is too nerdy on Oct 7, 2008 11:21 PM PDT up reply actions 0 recs

I think it's as close to real, legitimate scientific research that is presented in real time ...

which, granted, is a far cry from real political science — but, based on the standards you’ve laid out, I’d challenge you to offer anything that offers a more sophisticated, more scientific, more up-to-date analysis of the election.

The data is imperfect — fine, I agree. The results, therefore, are imperfect. Fine, I agree on that too. So what? Imperfection does not mean it isn’t valuable or accurate.

Like you said, it’s a blog, it is what it is. But it’s a lot more sophisticated, scientific and meaningful than any of the other trash that’s out there. It’s not a typical peer reviewed study — no — it’s a blog. There are no “typical, peer reviewed studies” on the election as it happen. Political science is history — his blog is journalism (from an analysis, rather than reporting angle). It’s more in depth, more sourced, more open in its methodology than anything you’d find in the New York Times or Time.

I mean, seriously, your standards are unrealistic, to the point of being irrelevant.

"It's for your own good. Big strong Devo knows whats best for Poppy" -- Mossback

by devo on Oct 8, 2008 12:06 AM PDT up reply actions 0 recs

Playing devil's advocate

I don’t know if you caught Silver on Colbert’s show last night, but he did, to a point, address the issue of bias in the numbers he’s collecting (not his own, if that exists in the data). While he didn’t get into the details or methodology, he did say that, essentially, he fixes (my word, not his) those issues.

I can see where the sketpicism comes in, but isn’t eliminating this sort of noise similar to eliminating a lot of the “luck” noise in figuring out a player’s offensive ability/contribution?

by thejd44 on Oct 8, 2008 7:46 AM PDT up reply actions 0 recs

I think we've reached a common ground here.

I agree the standards are too high, but all I’m saying is these kinds of predictions have so many holes and variables in them that they become quite unreliable. A lot of polling data leading up to the 2004 election indicated a slight Kerry victory or at least a tie, and then Bush ended up winning by a fair margin.

I’m just saying you can’t really rely on polling to give an accurate picture of an election because there are an incredible number of variables that can change the whole game. Look at what the sub-prime mortgage fallout has done for Obama in polling data, for example. Think about the effect Hillary’s crying had two days before she won in New Hampshire. A significant portion of the electorate is incredibly fickle and uninformed. I’d be interested to know exactly how many of them make their decision only days before the election.

I don’t expect polling data to hold up to the rigors of scientific research. By its very nature it cannot do that. My problem with the data, however, is how the journalists like to interpret it. They like to paint it as science on a nightly basis. I think that can be a little irresponsible.

So, I guess my point is that we have all this data, but is it really meaningful? I would guess not, but either way we won’t really know until months after the election…. at which point the media will have moved on and returned to using Lindsay Lohan or Britney Spears as the main nightly talking points.

by VORP is too nerdy on Oct 8, 2008 2:53 PM PDT up reply actions 0 recs

I'd certainly agree with the criticism of the media ...

You can’t blame the polls if they are misinterpreted. Bush won the 2004 election by 2.48% — inside the margin of error for any poll that had Kerry up by 2% or less. Those polls weren’t wrong, they were just misinterpreted.

In fact, check out the actual polls. Among the polls that RCP lists, every single poll, taken within roughly the last three weeks, was correct, within the margin of error. Given the confidence level, in fact, we should have expected that at least one poll in that period would have been wrong.

One big part of the reason that what Nate is doing is FAR more worthwhile than normal polls is because it is focused, primarily, on state polls — which makes the sample size, functionally, much larger — and it constantly regresses to the mean — which makes its projections far more conservative than normal poll interpretation.

You’re right that things can change dramatically. What Nate is projecting should be assumed to be, basically, this is what is the most likely outcome, assuming nothing too drastic happens between now and the election. It regresses to varying degrees, based on distance from the election, so it is appropriate to specify on election day, not if the election were held today.

In 2004, according to the exit poll, 11% of voters decided in the last week and about 80% had decided by this point in the cycle. The late numbers will likely shrink this time around, given the drastic increase in early voting over the last four years. Roughly one third of the electorate will have actually cast their ballot before the last week of the campaign — 50% more than in 2004.

Something drastic could happen, sure. If terrorists, the Russians or China attacks, McCain’s prospects will improve dramatically. If it turns out that Obama has an illegitimate child with a white woman, even more so. But it probably won’t. The basic dynamics of the election as they stand today will probably be the basic dynamics of the election four weeks from yesterday.

"It's for your own good. Big strong Devo knows whats best for Poppy" -- Mossback

by devo on Oct 8, 2008 3:31 PM PDT up reply actions 0 recs

It’s also worth noting that Nate weights the polls by how accurately they have predicted past elections and normalizes results by comparing states with similar demographic or electoral makeups. In that way, it’s very similar to PECTOA. One of the best ways to eliminate personal bias is to pick a model and run with it.

If you look at the percentage, it gives a distribution rather than saying “Obama wins.” Currently, it gives Obama about a 90% chance of winning. That also means there’s a 10% chance McCain comes from behind, and it gives the relative likelihood of each electoral vote distribution.

by MrIncognito on Oct 8, 2008 9:20 PM PDT up reply actions 0 recs

Value Over Replacement Poll?

by mikev on Oct 8, 2008 10:26 PM PDT up reply actions 0 recs

DING!!! DING!!! DING!!!

"It's for your own good. Big strong Devo knows whats best for Poppy" -- Mossback

by devo on Oct 8, 2008 10:29 PM PDT up reply actions 0 recs

kudos to devo and VORP, good discussion.

IMO, some of Nate’s website’s success has had to do with his rigor in updating the polling in timely manner and adding content every day. I mean, BP is also just a glorified blog itself, but it has found an appreciative audience (and built a somewhat successful business model) among those who wanted much more insight from baseball writers than was readily found in big media.

I think that its true that fivethirtyeight isn’t exactly academic research, though it could be provided as a data set in that academic context once the election is finished. This is the kind of stuff that campaigns have had to be doing in-house anyway (and I’ve done this before in a professional capacity), so putting it out there for public consumption and critique is already an interesting step.

Are there biases inherent in the way that information is presented? Yes. There is an element of gamesmanship that goes on between campaigns in spinning the way that numbers look relative to their predictability. Part of campaign strategy is optimization of resources, and public poll numbers are often used to publicly bluff the other side when a side believes that internal polling is painting a different picture.

There was a large difference during the primaries between Hillary Clinton’s campaign and Barack Obama’s related to the attention each gave to the way delegates were allocated. Obama’s team ran up the score in many Republican states and in caucuses, gaining large margins in pledged delegates of Clinton. Her side on the other hand seemed slow to realize that Obama’s stretegy was a severe threat to her chances to win, despite Clinton winning more large states. More than I had ever seen, non-wonks were drawn into evaluating the numbers, and time and time again the big media was much slower on the uptake than wonky bloggers.

by jakarta on Oct 8, 2008 7:29 AM PDT reply actions 0 recs

The kind of gamesmanship you refer to

is why I think Nate should have probably kept his identity secret when it came to fivethirtyeight. As far as I know, he’s worked exclusively with leftist groups in the past. Opportunist conservatives/Republicans would use that as an opportunity to attack the legitimacy of his data. Don’t you just love how, on nightly political discussion shows, you have two different so-called “experts” coming to wildly different conclusions based on their political persuasion?

by VORP is too nerdy on Oct 8, 2008 2:57 PM PDT up reply actions 0 recs

He specifically says in his FAQ that, while Illinois does not have partisan voter registration, he votes Democrat ...

I don’t think secrecy would have really helped anything …

"It's for your own good. Big strong Devo knows whats best for Poppy" -- Mossback

by devo on Oct 8, 2008 3:34 PM PDT up reply actions 0 recs

People attracted to empiricism are more likely to be swayed by criticism of the model rather than nonspecific charges of bias. The fact that someone supports a candidate does not ipso facto render the analysis inaccurate.

by MrIncognito on Oct 8, 2008 9:26 PM PDT up reply actions 0 recs