Is The Media Exaggerating Obama's Lead?

In the aftermath of the Democratic National Convention, the media has been afire with reports of a post-DNC convention bump for President Obama. Politico especially has been touting this narrative, openly sneering about the "Kerry-ization of  Mitt Romney" and touting story after story claiming that even within the Romney campaign, sources close to the candidate (studiously unnamed, as always) concede that Romney will probably lose.

This is harsh stuff, and those parts of it that haven't been denied (in colorful language) by the Romney campaign could ordinarily could be dismissed as typical media water carrying.

However, the reason no one is brushing off remarks like this is simple - the data would appear to be on the side of those making these dire predictions about Romney's electoral future, bias or no bias. Gallup's polling shows Obama up 6 among registered voters. CNN's most recent poll has Romney up 6 points. Those numbers don't look good.

But what if the polls are wrong? Certainly that's the question most disappointed supporters of an apparently losing politician find themselves asking. But in this case, there is evidence that the question is a valid one. So what if the polls are wrong? And how would you know they're wrong?

Rush Limbaugh thinks he's worked out the answer to both these questions, and his response comes via the Daily Rushbo:

Limbaugh's answer can be reduced down to the following argument: Yes, the polls are wrong, and you can tell because the crosstabs in the polls show a deeply skewed sample that favors Democrats by a wide margin, and undersamples independents by an even bigger margin. As Limbaugh himself puts it:

The CNN Opinion Research Corporation poll reports Obama at 52; Romney at 46.  Unskewed the data reveals a 53 to 45 percent lead for Romney.  If the poll were honest, Romney would be plus eight in the CNN poll.  Not making that up.  This new CNN poll, unlike many other analyzed, not only oversamples the Democrat voters, it massively under-samples independents, which produces a result that's more favorable to Obama.  "This survey's sample includes 397 registered Republicans and 441 registered Democrats. But the survey included a total of 822 registered voters, leaving only 37 independent voters at most."  Thirty-seven independent voters out of 822.  That is a massive under-sampling.  And in those 37, Romney's up 14.  It's stunning.

Limbaugh is quoting from a particular analysis of the CNN poll by conservative writer Dean Chambers, who broke the poll apart in a brutal op ed for the Washington Examiner. Chambers writes:

he latest CNN/ORC poll released todayshows a wider lead for President Obamathan the previous CNN/ORC poll but it is doubly skewed. It massively under-samples independents while it also over-samples Democratic voters. The CNN/Opinion Research Corp. poll official reports Obama at 52 to percent and Mitt Romney at 46 percent. Unskewed, the data reveals a 53 percent to 45 percent lead for Romney.[...]

The sample for the CNN/ORC poll includes 50.4 percent Democrats and 45.4 percent Republicans and appears to have only 4.2 percent independents. This means independents are under-sampled 25 percent while Democrats are over-sampled 12.1 percent. Both of those are larger variations in sampling than seen in most polls that are likewise skewed by such sampling variations.

And that's not the only poll Chambers has attacked. He has mathematically taken down a poll by Investors' Business Daily showing Obama up 2, a poll by Washington Post/ABC showing Obama up 1, the aforementioned Gallup Poll, as well as practically every other poll that's been conducted over the past few weeks. He also has argued that if the election was held today, Romney would win over 300 electoral votes, trouncing Obama.

Those conducting the polls would likely respond that their samples are weighted to reflect expected turnout in November. That, or they would argue that their samples are based on a 2008-era breakdown of party affiliation. Or, in the case of Gallup, the response would generally be that registered voters always skew differently from likely voters, so their poll should be taken with that in mind.

Obviously, it's difficult to falsify a prediction until the election, so one can only let that defense pass as evidence of what may look like either excessive optimism or excessive pessimism on the part of the polling company. However, in the case of weighting turnout to reflect 2008, these numbers can be checked. Which we will do for the two most recent polls.

CNN

According to CNN's exit polls from 2008, the turnout statistics were as follows: 39 percent of Democrats voted, compared with 32 percent of Republicans and 29 percent of independents. Using these numbers, the CNN poll, as well as the other polls that disproportionately favor Obama, can be checked for some statistical bias.

And indeed, if those numbers hold true, the situation does change. The recent CNN poll's sample would have yielded a 4 point advantage for Obama under those circumstances. That's a wide lead, but it's nowhere near the 7 point lead Obama had in 2008 with the same demographics. By this measure, therefore, the President's support can be said to be weaker at this point in the campaign than it was on Election Day 2008. However, it still isn't a Romney lead by a long shot, assuming 2008's totals hold true - not an entirely realistic assumption, given the different dynamics of this race versus the last one. A 7 point turnout advantage for Democrats, for instance, is quite unlikely.

However, even using the 2008 figures, the picture painted by the CNN poll comes off as highly skewed. CNN predicts not only a 5 point turnout advantage for Democrats, but predicts a fall of a staggering 23 points in turnout among independents, with no explanation as to why. That's a strike against that poll, by any measure.

ABC/Washington Post

Again, using the turnout figures from 2008, this poll is off by a substantial margin. By our calculations, 2008 numbers would also indicate a 4 point race in Obama's favor using the total number of voters Washington Post/ABC quizzed.

However, unlike the CNN poll, the different sampling on the part of the Washington Post/ABC is much more plausible, given that the poll projects decreased Democratic and Republican turnout, while sampling independents by a wider margin. The poll breaks down to 33 percent Democrats, 23 percent Republicans, and 37 percent independents. This arguably under-samples Republicans, but given that the poll weights independents (who it shows disfavor Obama by 11 points) so highly makes it less favorable to the President. And indeed, that is reflected by the poll's end result, which shows Obama with a minuscule 1 point lead. That isn't a convention bounce by any stretch of the imagination. And indeed, Rasmussen Reports' poll from today shows that that lead is probably where the race is right now.

So are they biased?

In the case of CNN, the bias looks difficult to debate. ABC/Washington Post, on the other hand, has a mildly under-sampled Republican electorate, but otherwise paints a potentially accurate picture. Bias is still very much something to be wary of, as Democratic pollsters Pat Caddell and Doug Schoen have argued, but it doesn't prevent an accurate depiction of the data in plausible scenarios.

However, as demonstrated with the CNN poll, not every scenario advanced by the media in this election is plausible, and those that aren't tend to yield the biggest outliers when it comes to statistical results. 2012 is still very much up in the air, and the rumors of 6 point leads on Obama's part should probably be taken with a grain of salt.