« Where Eagles Dare: A Message For Senator McCain | Main | When Will They Consummate It? »

State Poll Accuracy

I have spent a great deal of time analyzing faulty procedures in the national political polls for the presidential race this year. I have noted proofs of invalidity and the significance of party affiliation weighting in the published claims of such polls. But a number of people have observed that Barack Obama is also leading in a number of state polls, and they quite reasonably ask if those polls are valid or not. The chief difficulties in the state polls, are the facts that there are so many polling groups performing state polling (Real Clear Politics has reports from 49 different polling groups on state polls for the presidential race), there are different demographic norms for each state, and whatever is skewing the national polls is likely to be influencing reported state results as well. But while the past does not guarantee the future, I did find some interesting points in the 2004 state polls.

Not every polling group which is publishing polls this year also polled and published in 2004. And many of the polling groups are limited to only a few states, or just one. The Field Poll, for example, is only concerned with California. So I limited my analysis to nine major polling groups which conducted state polls in a large number of states in 2004. I was looking for polls which released polls just prior to the election, and which also released polls during the first two weeks of October of 2004. The results of that examination are presented here. I hope you will find them illuminating.

Survey USA

Survey USA conducted polls in 30 states just before the 2004 election. Of those 30 polls, in 8 state polls the published support for Bush was off by 3 points or more, as many as 6 points wrong. In 5 states Bush's support was over-estimated, in 23 states his support was under-estimated. In 5 state polls the published support for Kerry was off by 3 points or more, as many as 10 points wrong. In 10 states Kerry's support was over-estimated, in 19 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 14 times, off by 5 points or more in 6 states.

Survey USA conducted polls in 30 states during the first two weeks of October 2004. of those 30 polls, in 14 states support for Bush was off by 3 points or more, as many as 6 points wrong. In 3 states Bush's support was over-estimated, in 23 states his support was under-estimated, In 11 state polls the published support for Kerry was off by 3 points or more, as many as 11 points wrong. In 6 states Kerry's support was over-estimated, in 19 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 14 times, off by 5 points or more in 7 states.

- continued -

American Research Group (ARG)

ARG conducted polls in 7 states just before the 2004 election. Of those 7 polls, in 3 state polls the published support for Bush was off by 3 points or more, as many as 4 points wrong. In 0 states Bush's support was over-estimated, in 6 states his support was under-estimated. In 1 state poll the published support for Kerry was off by 3 points or more, as many as 3 points wrong. In 1 state Kerry's support was over-estimated, in 5 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 2 times, off by 5 points or more in 1 state.

ARG conducted polls in 49 states during the first two weeks of October 2004. of those 49 polls, in 40 states support for Bush was off by 3 points or more, as many as 11 points wrong. In 2 states Bush's support was over-estimated, in 45 states his support was under-estimated, In 23 state polls the published support for Kerry was off by 3 points or more, as many as 9 points wrong. In 8 states Kerry's support was over-estimated, in 32 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 28 times, off by 5 points or more in 24 states.


Rasmussen

Rasmussen conducted polls in 11 states just before the 2004 election. Of those 11 polls, in 3 state polls the published support for Bush was off by 3 points or more, as many as 5 points wrong. In 0 states Bush's support was over-estimated, in all 11 states his support was under-estimated. In 3 state polls the published support for Kerry was off by 3 points or more, as many as 5 points wrong. In 1 state Kerry's support was over-estimated, in 8 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 3 times, off by 5 points or more in 2 states.

Rasmussen conducted polls in 31 states during the first two weeks of October 2004. of those 31 polls, in 19 states support for Bush was off by 3 points or more, as many as 10 points wrong. In 2 states Bush's support was over-estimated, in 29 states his support was under-estimated, In 13 state polls the published support for Kerry was off by 3 points or more, as many as 10 points wrong. In 5 states Kerry's support was over-estimated, in 23 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 12 times, off by 5 points or more in 9 states.


Mason-Dixon

Mason-Dixon conducted polls in 21 states just before the 2004 election. Of those 21 polls, in 11 state polls the published support for Bush was off by 3 points or more, as many as 7 points wrong. In 0 states Bush's support was over-estimated, in all 21 states his support was under-estimated. In 14 state polls the published support for Kerry was off by 3 points or more, as many as 6 points wrong. In 1 state Kerry's support was over-estimated, in 20 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 6 times, off by 5 points or more in 2 states.

Mason-Dixon conducted polls in 19 states during the first two weeks of October 2004. of those 19 polls, in 11 states support for Bush was off by 3 points or more, as many as 7 points wrong. In 1 state Bush's support was over-estimated, in 18 states his support was under-estimated, In 15 state polls the published support for Kerry was off by 3 points or more, as many as 6 points wrong. In 2 states Kerry's support was over-estimated, in 17 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 8 times, off by 5 points or more in 4 states.


Zogby

Zogby conducted polls in 13 states just before the 2004 election. Of those 13 polls, in 9 state polls the published support for Bush was off by 3 points or more, as many as 6 points wrong. In 0 states Bush's support was over-estimated, in all 13 states his support was under-estimated. In 3 state polls the published support for Kerry was off by 3 points or more, as many as 6 points wrong. In 6 states Kerry's support was over-estimated, in 4 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 10 times, off by 5 points or more in 4 states.

Zogby conducted polls in 15 states during the first two weeks of October 2004. of those 15 polls, in 12 states support for Bush was off by 3 points or more, as many as 8 points wrong. In 1 state Bush's support was over-estimated, in 14 states his support was under-estimated, In 5 state polls the published support for Kerry was off by 3 points or more, as many as 11 points wrong. In 5 states Kerry's support was over-estimated, in 7 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 10 times, off by 5 points or more in 8 states.


CNN

CNN conducted polls in 6 states just before the 2004 election. Of those 6 polls, in 4 state polls the published support for Bush was off by 3 points or more, as many as 5 points wrong. In 2 states Bush's support was over-estimated, in 4 states his support was under-estimated. In 4 state polls the published support for Kerry was off by 3 points or more, as many as 6 points wrong. In 3 states Kerry's support was over-estimated, in 3 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 5 times, off by 5 points or more in 5 states.

CNN conducted polls in 11 states during the first two weeks of October 2004. of those 11 polls, in 4 states support for Bush was off by 3 points or more, as many as 4 points wrong. In 2 states Bush's support was over-estimated, in 7 states his support was under-estimated, In 6 state polls the published support for Kerry was off by 3 points or more, as many as 6 points wrong. In 2 states Kerry's support was over-estimated, in 9 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 9 times, off by 5 points or more in 3 states.


Research 2000

Research 2000 conducted polls in 9 states just before the 2004 election. Of those 9 polls, in 5 state polls the published support for Bush was off by 3 points or more, as many as 5 points wrong. In 0 states Bush's support was over-estimated, in all 9 states his support was under-estimated. In 0 state polls the published support for Kerry was off by 3 points or more. In 2 states Kerry's support was over-estimated, in 7 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 4 times, off by 5 points or more in 2 states.

Research 2000 conducted polls in 12 states during the first two weeks of October 2004. of those 12 polls, in 11 states support for Bush was off by 3 points or more, as many as 7 points wrong. In 1 state Bush's support was over-estimated, in 11 states his support was under-estimated, In 5 state polls the published support for Kerry was off by 3 points or more, as many as 6 points wrong. In 2 states Kerry's support was over-estimated, in 8 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 6 times, off by 5 points or more in 5 states.


Quinnipiac

Quinnipiac conducted polls in 3 states just before the 2004 election. Of those 3 polls, in 1 state polls the published support for Bush was off by 3 points or more, as many as 3 points wrong. In 0 states Bush's support was over-estimated, in all 3 states his support was under-estimated. In all 3 state polls the published support for Kerry was off by 3 points or more, as many as 5 points wrong. In 0 states Kerry's support was over-estimated, in all 3 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 1 time.

Quinnipiac conducted polls in 5 states during the first two weeks of October 2004. of those 5 polls, in 0 states support for Bush was off by 3 points or more. In 1 state Bush's support was over-estimated, in 2 states his support was under-estimated, In 3 state polls the published support for Kerry was off by 3 points or more, as many as 12 points wrong. In 0 states Kerry's support was over-estimated, in all 5 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 3 times, off by 5 points or more in 1 state.


Fox News

Fox News conducted polls in 4 states just before the 2004 election. Of those 4 polls, in 1 state poll the published support for Bush was off by 3 points or more, as many as 8 points wrong. In 0 states Bush's support was over-estimated, in all 4 states his support was under-estimated. In 2 state polls the published support for Kerry was off by 3 points or more, as many as 5 points wrong. In 1 state Kerry's support was over-estimated, in 3 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 3 times, off by 5 points or more in 1 state.

Fox News conducted polls in 3 states during the first two weeks of October 2004. of those 3 polls, in 1 state support for Bush was off by 3 points or more, as many as 4 points wrong. In 0 states Bush's support was over-estimated, in all 3 states his support was under-estimated, In all 3 state polls the published support for Kerry was off by 3 points or more, as many as 5 points wrong. In 0 states Kerry's support was over-estimated, in all 3 states his support was under-estimated. The margin between the candidates was off from actual election results by 3 points or more 1 time.


SUMMARY

Even this brief examination reveals that the state opinion polls were often wrong, and often to a materially significant degree. In 2004, both Bush and Kerry's support were commonly under-estimated, but in several groups' work, all of Bush's support was underestimated and Kerry's support was much more likely to be over-estimated. The margin was also often wrong, and to a significant degree. State polls, therefore, are not proven to be a valid predictor of election results.


[ the dominant source for this examination was the archive of 2004 state polling results, provided by Real Clear Politics]


TrackBack

TrackBack URL for this entry:
/cgi-bin/mt-tb.cgi/32156.

Listed below are links to weblogs that reference State Poll Accuracy:

Comments (60)

Thank you DJ, I always hate... (Below threshold)
epador:

Thank you DJ, I always hated statistics and analysis. Your thorough works (as well as ready appropriate response to valid criticism) are appreciated.

DJ,i'm heartened b... (Below threshold)
Halverson:

DJ,

i'm heartened by your analysis, but what do you make of Intrade and other betting sites that show an Obama victory. i've heard that these sites are extremely accurate at predicting the winner. would appreciate your thoughts.

Interesting work, DJ.... (Below threshold)
_Mike_:

Interesting work, DJ.

Intrade is not a predictor ... (Below threshold)
DJ Drummond:

Intrade is not a predictor of anything, Halverson. For proof, just look at the recent past there. Obama trailed Clinton for a very long time on the trading sites during the primaries, until he took control, and then the site's tracking fell in line afterwards. Same here, the trading site is simply an informal and unscientific poll of its own, which reflects only the current mood of its participants. It chases the polls, but is useless as a predictor.

That's the key about any poll, though - even the best and most accurate can only tell you the present mood, not what will happen in the future.

I would also note, that if the polls suddenly started showing McCain gaining strength, that his numbers on Intrade would also rise. Because many of the people there are just watching the polls, not the issues or analyzing the raw data.


DJDo you have a sp... (Below threshold)
Larry:

DJ

Do you have a spreadsheet? If so, could you make it available?

It's a bit unweildy, Larry.... (Below threshold)
DJ Drummond:

It's a bit unweildy, Larry. 50 states and 9 polling groups on 2 sets of dates for 3 catagories of comparison is a tight fit on any screen.

Thanks DJ. Always an inter... (Below threshold)
Ray H.:

Thanks DJ. Always an interesting read when you dive into the analytics of the polls.

Thank you, DJ for your resp... (Below threshold)
Halverson:

Thank you, DJ for your response. i appreciate it. i'm looking forward to casting my ballot for the only candidate that puts his country above all else.

Please keep up your excellent work and analysis because i fear that a lot of Republicans are dispirited by the MSM and the polls.

Everytime i see a poll showing an Obama lead/victory i remind myself that President Gore and President Kerry would disagree with the polls.

DJIf I somehow giv... (Below threshold)
Larry:

DJ

If I somehow give you my email address, would you be willing to send it to me?

Wait. This post has been u... (Below threshold)
Sheik Yur Bouty:

Wait. This post has been up for an hour and there's been no rebuttal by mantis????!!!??

Is he feeling OK?

I hope he shows up. ... (Below threshold)
Larry:


I hope he shows up. I am still trying to get him to respond to a free speech thing about which he is in denial.

Send me an email Larry (dru... (Below threshold)
DJ Drummond:

Send me an email Larry ([email protected]), and I will send you my information as an email attachment.

DJ

Sent - and a public ... (Below threshold)
Larry:


Sent - and a public thank you DJ.

Polls are snapshots, not pr... (Below threshold)

Polls are snapshots, not predictions. A poll on October 1st isn't a prediction of what the outcome of the election will be a month later. It is a statement on where the election stands on October 1st.

Also, you don't mention margin of error. A poll that is within 3 to 5 points is actually very accurate.

Further, the electoral college is a zero sum game. If a poll predicts a candidate to win by one and that candidate wins by six then the only thing the pollster got wrong was the margin of error.

Anyone who doesn't know these simple things about polling shouldn't be giving advice on polling.

Ahhh, I am so glad you ment... (Below threshold)
DJ Drummond:

Ahhh, I am so glad you mentioned margin of error, Blue.

Statistics for opinion polls uses a confidence interval of 95%, which in simple words means that if you had 20 different groups do the poll at about the same time, 19 of them would get the same results, within in the margin of error.

Now, these are the 10 national polls punched up on Real Clear Politics:

Democracy Corps - O + 3
Battleground - O + 3
CBS News - O + 3
Reuters - O + 4
Rasmussen Tracking - O + 5
Hotline FD Tracking - O + 6
NBC News - O + 6
Ipsos - O + 7
CNN - O + 8
Gallup Tracking - O + 11

Now, these polls are claiming a margin of error of 3 percent, meaning that to be valid, at least 9 of 10 must be within 3 points of each other. It's not hard to see that this is not the case.

Therefore, the math alone PROVES the methodology used in these polls is INVALID.

As to the margin of error's significance, you ignore the obvious fact that when a poll predicts Candidate A will win by 4 points and Candidate B instead wins by 2, that 6 point error is, as I said, materially significant.

As to the October polls, the noise being made right now is about the early October polls. My point was quite clear about what those polls are worth as predictors, I'm so glad you finally grasped what I was saying all along.

It's awesome that you're do... (Below threshold)
Clint:

It's awesome that you're doing this work.

But you're looking at the wrong statistic.

Of course the polls systematically undercount the support of both candidates.

The polls include an "undecided" category. The voting booth does not.

I'd love to see how the numbers come out if you go back and look at the difference between the support for the two candidates in the polls and the difference between their actual vote totals in the election.

The former are the numbers we really obsess over with the polls and the latter are what matters -- and what we're trying to predict.

Having said all that... the fact that so many polls actually over-estimated support for Kerry is a pretty damning indictment.

Important to remember this as we panic over the supposed gap in the polls.

Polls are not accurate. Eve... (Below threshold)
Hestrold:

Polls are not accurate. Even the election results are not accurate. Remember Florida in 2000. Anytime you analyze something you have an effect on the data you are looking at. Hey, kinda like quantum physics. And personal and institutional bias colors (oh can I say that word anymore?) the results. The fact that we fixate on polls is the 'instant gratification' world we live in.

If they couldn't beat Georg... (Below threshold)
Pretzel Logic:

If they couldn't beat George Bush, G-e-o-r-g-e B-u-s-h! with John Kerry, what makes them think Barry can beat John McCain??

Statistics for opi... (Below threshold)
Statistics for opinion polls uses a confidence interval of 95%, which in simple words means that if you had 20 different groups do the poll at about the same time, 19 of them would get the same results, within in the margin of error.

That is not what it means. You can educate yourself here if you really want to know what it means.

As to the margin of error's significance, you ignore the obvious fact that when a poll predicts Candidate A will win by 4 points and Candidate B instead wins by 2, that 6 point error is, as I said, materially significant.

As someone else noted, polls include undecided voters, elections do not. As a result a six point swing isn't materially significant. That is one reason why sites that aggregate polling data call leads of less than 5% a toss up.

You are misinforming people about the basics of polling.

Thanks Blue, I will trust m... (Below threshold)
DJ Drummond:

Thanks Blue, I will trust my stats professors and the texts I consulted, rather than an MSM argument in a magazine. You're awfully dishonest and cranked in your comments, I guess you don't like folks finding out "The One" is not everyone's 'Messiah', but just a 'mess' to many of us.

2FB.

Blue: "You are misinform... (Below threshold)
DJ Drummond:

Blue: "You are misinforming people about the basics of polling".

Not in the least. You, however, are lying.

The first rule of empirical validation is reproducability. If a claim is valid, the test results can be reproduced. That is expressed in statistics by the confidence interval, which is why I brought it up.

I could punch out a detailed explanation of a two-tailed t-test, wherein the confidence interval (or its inverse, the statistical significance level) is used to test for a "Type I error", also called a "false positive determination", but really, the readers here just want to know what the published claims of a poll mean, and if you work with opinion statistics at all, you understand that the ability to reproduce a poll and get the same results within the margin of error are, in fact, the most meaningful verification of internal validity of the method used and the data chosen.

The only question remaining is whether you understand the subject and are lying because you do not like the proof that the opinion poll methodology used is invalid, or you do not understand the subject, and are trying to attack what you do not understand.

The first rule of ... (Below threshold)
The first rule of empirical validation is reproducability. If a claim is valid, the test results can be reproduced. That is expressed in statistics by the confidence interval, which is why I brought it up.

Polls aren't experiments. They aren't meant to be reproduced.

Blue: "Polls aren't expe... (Below threshold)
DJ Drummond:

Blue: "Polls aren't experiments"

Technically, that's exactly what they are. Human behavior analysis, or did you forget that part?

You're awfully dis... (Below threshold)
You're awfully dishonest and cranked in your comments, I guess you don't like folks finding out "The One" is not everyone's 'Messiah', but just a 'mess' to many of us.

I am not dishonest. I am annoyed though. I was originally going to ignore this story of yours because I thought it was just read meat polemics for the base, but then I realized you are making people more confused about an issue that will outlive the election. Stick to calling Obama a terrorist. You are embarrasing yourself re: the polls.

Riiiiight, mister 'a... (Below threshold)
DJ Drummond:

Riiiiight, mister 'a poll is not a test'. You keep believing that.

I am confusing no one, and despite your attempts, neither are you.

Obama's folks have been claiming the election is over, because the polls show them ahead. However, there are fundamental errors, critical flaws, in those polls, demonstrated by the evidence I have presented here and before. In a series of articles, I have discussed the history of opinion polling, the structure and methodology, the authorities (NCPP and AAPOR), demonstrated the mechanics of weighting and shown how to reach the raw data behind the published claims.

I discussed the number of undecideds and their importance to the election several times in the past month, but you ignored that.

I have shown why polls can be valid even if you or I disagree with their claim, but you missed that one too.

So today, I continued with fulfilling a request made of me long-ago, and showed the statistical value of the October and final state polls from 2004. You could not even manage to understand that the whole purpose of an opinion poll is to test the claim made that a certain level of support exists for a candidate.

Your lies have gotten so bizarre Blue, that you not only ignored what I have been writing about for two months, you lied and claimed I was calling Obama a "terrorist", something I have never done, and to the best of my knowledge has never been asserted by any of the Wizbang editors or writers.

I am in no way embarrassing myself. It's just as well, however, that you are using a pseudonym, as if you were using your real name your family would be very much embarrassed by your spit-faced rants.

A few questions...... (Below threshold)
ORyan:

A few questions...

How do you define "just before the election"?

Why don't you condense this data into some sort of single rating for these pollsters or try to identify and determine a polls lean? If they lean reliably in one direction then the poll would appear to have a lot of predictive power if you correct for the lean.

Also, how can somebody argue a poll is not an experiment; particularly ones close to an election? It is observation (the polling part), a hypothesis/prediction (the poll's result), and an experiment (the election). Polls attempt to make predictions which will be verified/falsified on election. That is pretty scientific.

ORyan, polls released Octob... (Below threshold)
DJ Drummond:

ORyan, polls released October 29 or later were "just before the election", which was November 2.

The problem with a single rating, is that it attempts to quantify a whole range of qualitative data. It would be hopelessly subjective. For instance, is it more significant to get the margin correct, or the level of support for a given candidate? Is it significant if there is a lot of movement in a poll between dates, or is stability a better indicator of a "good" poll?

I know my posts are long, but I want the reader to have as much useful information as possible, in order to help them make up their own minds.

ORyan - That is th... (Below threshold)
JLawson:

ORyan -

That is the sceintific method - obervation/hypothesis/experiment/observation of results and determination if they match hypothesis.

But what if the methodology isn't designed to produce a valid predictive poll, but is instead designed to push the numbers in a direction they wouldn't normally go? There've been reports of the samples being skewed towards Democrats by 15% or more and yet Obama's up by only 12 points. Is the sample a random one, or is the skewing deliberatly toward one direction?

Goes to show the only really valid polls are going to occur on 4 Nov - anything else up until then is sheer speculation and entertainment.

Sorry, DJ - should have wai... (Below threshold)
JLawson:

Sorry, DJ - should have waited for your reply.

Your lies have got... (Below threshold)
Your lies have gotten so bizarre Blue, that you not only ignored what I have been writing about for two months...

What lies? Your polling analysis is just plain dumb. Please list my lies and I will try to educate you about your misunderstanding. I can't promise anything though.

wow. Blue is dishonest ... (Below threshold)
DJ Drummond:

wow. Blue is dishonest and stupid.

If more of Obama's folks are like Blue, we can stop worrying, they'll screw up their votes and accidentally vote for Buchanan again.

wow. Blue is disho... (Below threshold)
wow. Blue is dishonest and stupid.

Don't be afraid. Tell me about the lies. I triple dog dare you.

"I triple dog dare you."... (Below threshold)
JLawson:

"I triple dog dare you."

LOL. Oh, THAT'LL make people take you seriously!

DJYou are wasting ... (Below threshold)
Larry:

DJ

You are wasting your time with blue. He hangs on a site that is upscale Obamanut, with pretensions of intellectual honesty. Those folks you can never move off of their ignorance because they don't understand their problem.

This is the same type as those who signed the support for Bill Ayers thing. They just don't get it and likely never will.

Maybe you already knew that.

I knew that, Larry. Blue's... (Below threshold)
DJ Drummond:

I knew that, Larry. Blue's just fun to tweak.

I knew that, Larry... (Below threshold)
I knew that, Larry. Blue's just fun to tweak.

You and Larry are welcome to hang out at my "Obamanut" blog. I don't think you would last long though. We expect people to defend their arguments.

Awwww, poor Blue is lonely.... (Below threshold)
DJ Drummond:

Awwww, poor Blue is lonely. But no thank you Blue, as much as I appreciate the invitation, my wife expects that I stay away from places of ill repute.

You think I'm rough on you, you don't want to know the wife's opinion of Mister Obama and his, hmm, ilk.

Mr. Drummond:More ... (Below threshold)
Andrew Byler:

Mr. Drummond:

More can be found by examining voting trends over the last 8 elections in states and then thinking about what we are seeing the candidates do on the ground as well as what the polls are alleging to tell us.

For example, in Indiana, we are being told the race is 49 McCain, 46 Obama. In the past 8 elections, the Republicans have outperformed their national popular vote by an average of 6.2% +/-3%, with the recent trend being to a higher outperformance (Bush outperformed the national result by 9.2% in Indiana). Meanwhile, the Democrat percentage has been shrinking from an average of a 3.5% underperformance in 1976 to 1984 to a steady decline to a 9% underperform in 2004. In short, Indiana averages a 13-14% favor towards Republicans vs. the national result.

If McCain is really only going to win Indiana by 3 or 4 points, this implies that Obama is going to win nationally by 10%, which would make the race 55% Obama, 45% McCain.

Similarly, polls are alleging North Carolina is tied. North Carolina has favored Republicans by 6.4% +/-2% in the last 5 elections, while the Democrats have lost by 5% +/-0.3% in the last three elections. Again, the implication is Obama is going to win by 11%+ nationally if the state poll is correct.

However, we don't see such polls in Georgia. Georgia has also been a Republican overperform by 6.5% Democrat underperform by 5.5% in the last 5 elections. McCain is said to be a solid 9-10% ahead there. If that is the score there, he's tied, up 1, or back just 1 or 2 nationally. It seems pretty unlikely that Obama would have such great attraction in Indiana and North Carolina that those races would be close or tied, but would be 10% behind in Georgia.

Now, since 1944, the Democrats have won over 49.8% of the total vote exactly twice - the 1964 Johnson landslide over Goldwater with a 61.1% share, and the Carter edging of Ford with a 50.1% share. ANY POLLING CLAIM THAT THE DEMOCRATIC CANDIDATE IS GOING TO TAKE 51% OR MORE OF THE TOTAL VOTE MUST BE GREETED WITH EXTREME SKEPTICISM. This is based on the fact that in the past 15 elections, they've only done it once. Going back even further, apart from the 4 Roosevelt elections in the Great Depression, the Democrats last won more than 50.1% of the vote in 1876, when they got 50.9%. In other words, since the Civil War, the Democrats have won over 50.1% in a national election just 6 times, 4 of them during the Great Depression.

On the other hand the Republicans have taken a 50%+ share of the vote in 7 of the last 15 elections. They also won 49.6% in 1960, 48% in 1976, and 47.9% in 2000. The only elections the Republicans have not won at least 48% of the vote, except 1964, have involved major 3rd party candidates - 1948 Strom Thurmond, 1968 George Wallce, 1992 and 1996 Ross Perot. Again going back to 1864, the Republicans have won an outright majority (not a plurality but a majority) in 17 of 36 elections. There is, of course, no major 3rd party candidate this year.

Therefore, I assert that McCain will do no worse than get 48% of the vote, because no polls of any sort show Obama producing a 20%+ blowout like 1964, and also that Obama will do no better than get 50% of the vote, with the remaining 2% going to minor party candidates.

If McCain only gets 48%, he would very likely lose Ohio, Florida, Missouri, Nevada, Iowa, New Mexico, and Colorado. That would be enough to hand the election to Obama.

However, McCain's campaign seems to believe that they have a real chance in Pennsylvania, Wisconsin, Iowa, Minnesota, New Mexico, and New Hampshire as well as holding almost all the other 2004 Bush states. OTOH, they seem to have given up on Michigan. Lets look at these.

Michigan has been trending more and more Democrat since 1984 to the point where it is now a 3%+ Democrat outperform, while Republicans have been underperforming by 2% there in 5 of the last 8 elections. The decision not to spend money there would seem to say McCain sees himself as ahead by less than 5% nationally - in other words something like 52-48 or 51-49 or 50-50 ignoring minor parties.

This would explain the continued focus on the upper midwest and Pennsylvania and New Hampshire. Wisconsin and Pennsylvania are states where the Republican underperforms by 1.3 or 1.4%. New Hampshire is a 0.7% underperform. Iowa and New Mexico are 0.1 to 0.2% underperforms for Republicans. Minnesota is interesting in that it has trended from 8% Republican underperform 1972 to 1988, to 5.7% underperform in 1992 and 1996, to 2.5% underperform in 2000 and 2004. The implication of the trend is Republicans will run even there in this election, which is also being born out in some polls showing a tie or a 1% lead either way. Meanwhile, the Democrats have averaged a 1.2% overperform in Minnesota int he past 4 elections. If the polls are really tied there or within 1%, it implies the national race is tied or McCain is ahead by up to 2%.

The McCain campaign is behaving as though this election is similar to the ones in 1960, 1968 (without Wallace taking votes from Nixon and Humphrey), 1976, 1980 (without Anderson taking votes from Carter), 1996 (without Perot taking votes from Dole), 2000 (without Nader and Buchanan), and 2004, or perhaps a bit better. They seem to think McCain will get 48-52% of the vote and pull off a win in the electoral College of somewhere between 270 to 330 votes.

The Obama campaign seems to think something nearly historically unprecedented is going on, and that their candidate will win by 10% or more with an outcome like 55-45% or 56-44%.

I would ask the reader which is more likely. The same old narrow election split we've seen so many times in this country, or a runaway Democrat landslide of 10+% in the popular vote with Democrats getting up towards 375 or 380 electoral votes?

Nice analysis, Andrew. And... (Below threshold)
DJ Drummond:

Nice analysis, Andrew. And it reminds me of a point I always want to make about polls - if a movement is true, there must be a cause. When a state or national poll changes for no apparent reason, this is a red flag for the poll's methodology.

DJ, thanks for a very educa... (Below threshold)
PVD:

DJ, thanks for a very educational post. Andrew, thanks for a terrific contribution as well.

Therefore, the mat... (Below threshold)
Clint:
Therefore, the math alone PROVES the methodology used in these polls is INVALID.

15. Posted by DJ Drummond | October 9, 2008 1:08 PM

I think you've made a factor-of-two error.

The 95% margin of error means that (ignoring systematic effects -- like slanted weighting-by-party-ID) 19 times out of 20 the TRUE value of the measured quantity will be within +/- 3% of estimated value. This means that those 19 estimates will have a range of 6%, not 3%.

Given the polls you cite, I'd guess the "true" value (again, minus the significant skew thrown in by the party ID assumptions, and other such effects) is about O+5. Then any of the polls between O+2 (-3%) and O+8 (+3%) are within the margin of error. The Gallup poll, finding Obama up by 11%, is almost certainly an outlier.

In point of fact, the way in which the polls AGREE is surprising to me, given the way some of them are slanting their party ID weights. That indicates to me that they are all doing something similar.

That means that either something completely unprecedented in modern history is happening -- the death of the Republican Party, which is exactly what it would mean to have a 15% deficit in party members actually voting -- or all of the pollsters are getting the race entirely wrong.

Given the plethora of historical examples of the latter occurring.... I agree with Andrew Byler's conclusion. This is a much closer race than anyone is reporting.

Andrew Byler-Great... (Below threshold)
Clint:

Andrew Byler-

Great post!

I completely agree with your conclusions, but it's worth noting that there are good reasons for some of the states to be behaving differently than in 2000 and 2004.

Iowa: McCain's patented "Straight Talk" means he's one of the only Presidential candidates to refuse to pander on ethanol. That's probably worth 10% of the vote in Iowa, unfortunately.

Indiana: Northwestern Indiana is basically suburban Chicago, so you should think of Obama as getting part of the boost there that any candidate gets in his "home" state. Of particular importance, it's the same "media market" -- so they got the ads from his Senate campaign (where he was running against Keyes, recall) and every time he was mentioned on the local news.

Interesting and informative... (Below threshold)
MPR:

Interesting and informative analysis gentlemen. I have been voting in elections since 72'(just dated myself). We have been at war before during an election. We have had financial crises during an election. We have had energy problems during elections. We have had Presidents that were popular and some not running for re-election. The same with Congress.
This truly is the most important election I have participated in and one where we have the convergence of of so many factors. This will really test the country's fortitude.
I have belonged to two unions. I am in business for myself. I've been a Republican and Democrat and voted across party lines at times.
This country is basically right of center and a far left liberal has not been elected as President before.
The mood of the country is different this time than it has been.
I hope the majority this time at least believes in prayer.

Actually no, that's not rig... (Below threshold)
DJ Drummond:

Actually no, that's not right Clint. That's because the margin of error cited by the polls is for the specific margin between the candidates, which is why they cite thew 3% figure instead of 6%.

I agree with you, though, that since the actual survey counts relative support for each candidate, which could be higher or lower by the margin of error than their report indicates, that the 6% MOE is the true margin. But since they publish a 3% MOE, I hold them to that. So, either the polls are invalid because the MOE does not cover 95% of polls using the same method in the same date range, or they are lying to say their margin of error is only 3%. Either way, they are caught out.

Nice work. However, how ma... (Below threshold)
mg:

Nice work. However, how many calls statewide were wrong? Not off by 3 percent or whatever, but predicted the wrong winner? Seems like that would be more relevant than predicting the exact percentage.

Good work DJ. I can't say I... (Below threshold)

Good work DJ. I can't say I understand all of it, but you do a great job laying out all of the data in terms that are within the grasp of the stat challenged like myself.

Clint:The worst un... (Below threshold)
Andrew Byler:

Clint:

The worst underperform for a Republican in Iowa was Bush I in 1988, where he was 8.9% lower in the Iowa vote than nationally. This is a singular result in the state's history since 1972, when it departed from its previous history of being a state with a large typical Republican overperformance.

1972 saw a 3.1% underperform
1976 was a 1.5% overperform
1980 was a 0.6% overperform
1984 was a 5.5% underperform
1988 was a 8.9% underperform
1992 was a 0.2% overperform
1996 was a 0.8% overperform
2000 was a 0.4% overperform
2004 was a 0.8% underperform

Until proven otherwise, the trend is always your friend.

If McCain is down 10% in Iowa, its highly likely he is also down 10% nationally. If the race is close, Iowa is probably also close.

It is entirely possible the race is both close and McCain is down 10% in Iowa because of a factor like ethanol. But it is not in conformance with most recent performances, and is therefore an outlier result. The McCain campaign itself has said it belives the race is close in Iowa. This implies that either their poll or the public poll is wrong, or the race nationally is much closer because of screwy turnout models in the public polls.

Another question to be addr... (Below threshold)
Andrew Byler:

Another question to be addressed is the overall turnout level of the electorate.

The long term trend going back to 1952 would suggest a raw turnout of around 128 million voters. However, many elections have fallen short of the trendline, and none have exceeded it by much. 1956, 1988, 1996, and 2000 were all well below the long term trend. The only years on or near the trend were 1960, 1984, 1992, and 2004.

I'm going to predict we don't hit the trend two elections in a row, and that we certainly don't exceed it.

The polls are so poor that ... (Below threshold)
Thomas Jackson:

The polls are so poor that if they were a tool, plumbers would use their hands rather than risk employing them.

I've been around long enough to remember 1968, 1972, 1980, 1984, 1988, 1994, and guess what? The pools were wrong on all of these. In some cases so badly wrong that one could only conclude they will willingly blind. Reagan vs Carter was a real contest? Anyone remember the great democrat landslide of 1994? Or how close the 1988 race was?

Obama will be buried in November and the MSM will tell us how racist we are, not what incredibly bad candidate Obama was or how radical he is.

DJ,Great analysis. ... (Below threshold)
Chris:

DJ,
Great analysis. This year is different in many ways but I have suspected the polls at all levels were missing something these last few weeks. I made the comment here a week or so ago about state polls being notoriously wrong/biased. My comment was more instinctive than analytical. Many years watching polls/politics and seeing them get it wrong so often. And the errors were/are almost exclusively in the same direction (Dem outperformance to recorded actual results).

Hopefully that will be the case again this year but even if that's the case, we're screwed as a nation for a long time. Not getting into that here but the polls having been so wrong will play a large role in the ensuing calamity. Hopefully I'm wrong on that.

Andrew,
Great contribution as well. It is possible that we'll get a Dem blowout but the odds are statistically slim. We are a center-right nation.

DJ,Is it possible ... (Below threshold)
Jeff:

DJ,

Is it possible that the real motive for all the phony voter registrations by ACORN is simply to skew the party weighting numbers ? 50k or 100k extra in a state could move the %. With skewed party weights the appearance of an Obama landslide might cause the GOP voters to stay home on election day. Maybe the ACORN folks don't actually expect to have phony voters try and use the phony registrations.

I believe, DJ, you said tha... (Below threshold)
Tola:

I believe, DJ, you said that the polls changed for "no apparent reason."

If I recall correctly, didn't they start to favor Obama during the beginning of the economic disaster?

And what also, of Gallup Po... (Below threshold)
Tola:

And what also, of Gallup Poll's credibility?

Jeff #51:If you ar... (Below threshold)
Andrew Byler:

Jeff #51:

If you are familiar with turnout in large cities, you will know that the absolute worst turnout numbers come from the poorest neighborhoods where ACORN and Co. are busiest registering "voters".

In my city of Philadelphia, for example, turnout is 70% in the middle class and upper class parts of the city, both black and white, and down around 25-35% in the poorest parts of the city, again both poor black and poor white neighborhoods.

Part of the reason for this is probably ACORN, and the other reason is inactive registrations due to the voter moving that cannot legally be purged from the roles for 4 years. The poorest neighborhoods are leaking residents to the middle class neighborhoods, which in turn are losing residents to the suburbs and other states.

It was recently in the news that Indianapolis now has a 105% registration of eligible voters. Philadelphia is somewhere around there too. But of this cumulative number of registrations, its unlikely that more than 60% will come to vote because so many of the registrations are inactive (or ACORN style fraud).

A large number of these "new voter registrations" are duplicates of people who have moved.

Interesting article here ab... (Below threshold)
Andrew Byler:

Interesting article here about Gallup in 2000.

http://www.huffingtonpost.com/alan-abramowitz/remembering-gallups-wacky_b_117594.html

Gallup poll numbers have swung more wildly than any other poll. I personally think the whole "convention bounce" storyline is a bunch of nonesense, as is the "scandal changing votes" storyline. I've met very few people who suddenly change their minds with the airing of each convention. They should be taken with a grain of salt.

The main factor in results is not wild swings of opinion among people in the middle, but resolution and effort to turnout the base vote in larger numbers than your opponent. If one side's base is depressed or fired up, this is what generally causes changes in results from the basic storyline of American elections.

Here's Gallup in 1996.

http://library.law.columbia.edu/urlmirror/CLR/100CLR524/ptpreselec.html

Notice that they consistently overestimated Clinton's support by 2-6% and got Perot right, and underestimated Dole's support by 3-8%. So they were "only" off by 5-14% on the final margin.

Well, the estimation for vo... (Below threshold)
Tola:

Well, the estimation for voter turnout is decreasing each year. This year, it is estimated to be less than 50%.

Most of the voters assumed NOT to turn up are poor people and college students - both groups which are in support of Barack Obama.

The Republican base is fired up right now - they simply cannot have Obama in office and hate what the media is doing. Do you think THIS will alter the election?

Ummm, 50% of what, Tola? T... (Below threshold)
DJ Drummond:

Ummm, 50% of what, Tola? Turnout in 2004, after all, was exceptionally high for both parties.

50% is not a high number.</... (Below threshold)
Tola:

50% is not a high number.

GREAT JOB AS USUAL DJ !!!!!... (Below threshold)
PaRep:

GREAT JOB AS USUAL DJ !!!!!!

Again Tola, 50% of wh... (Below threshold)
DJ Drummond:

Again Tola, 50% of what?




Advertisements









rightads.gif

beltwaybloggers.gif

insiderslogo.jpg

mba_blue.gif

Follow Wizbang

Follow Wizbang on FacebookFollow Wizbang on TwitterSubscribe to Wizbang feedWizbang Mobile

Contact

Send e-mail tips to us:

[email protected]

Fresh Links

Credits

Section Editor: Maggie Whitton

Editors: Jay Tea, Lorie Byrd, Kim Priestap, DJ Drummond, Michael Laprarie, Baron Von Ottomatic, Shawn Mallow, Rick, Dan Karipides, Michael Avitablile, Charlie Quidnunc, Steve Schippert

Emeritus: Paul, Mary Katherine Ham, Jim Addison, Alexander K. McClure, Cassy Fiano, Bill Jempty, John Stansbury, Rob Port

In Memorium: HughS

All original content copyright © 2003-2010 by Wizbang®, LLC. All rights reserved. Wizbang® is a registered service mark.

Powered by Movable Type Pro 4.361

Hosting by ServInt

Ratings on this site are powered by the Ajax Ratings Pro plugin for Movable Type.

Search on this site is powered by the FastSearch plugin for Movable Type.

Blogrolls on this site are powered by the MT-Blogroll.

Temporary site design is based on Cutline and Cutline for MT. Graphics by Apothegm Designs.

Author Login



Terms Of Service

DCMA Compliance Notice

Privacy Policy