« A Real Scandal | Main | Breaking: Obama Supporter Attacks and Mutilates McCain Volunteer »

Cooking Polls - State Poll Edition

My articles on polling have received a great deal of discussion. Most of it has been emotional in focus rather than factual, and I am amused by those blogs which reference the articles but ignore the context and substance in order to attack something I never even said. There have been a few reasonable questions, however, and one of them addresses the state polls. The national polls are all over the place, but what about the state polls? Don't they show Obama leading in most states, and don't the state polls basically agree? Those questions are good ones, and so state polling is the focus of today's article.

The first thing that jumps out at you if you read the state polls, is that there are a lot more polling groups doing polls at the state level, than at the national level. Also, most of the polling groups which do national polls, do not also do state polling, probably because it is expensive and difficult to try to cover all of the states on a consistent and timely basis. I have written before that national polls often focus on urban centers, which means that many of the states would require a functionally different methodology to work than what is used nationally. State polling tends to be smaller in respondent pool size, smaller in budget, and less frequent. Some polling groups only do one poll for the whole campaign, and it's common for even major groups to do a poll only once a month.

So anyway, I'm looking at the state polls and I notice that there's there's quite a range of opinion there, just like the national polls. California, for example, is all about Obama, but it ranges from +24 down to +16, which is statistically significant. No, it hardly means Cali is in play, but that degree of variance in a deep blue state indicates on the state level much of what I have been noting on the national level. Moving ahead alphabetically, Colorado looks pretty stable, but on the other hand RCP only shows a single poll done there all month. Something to think about, that. Even the Obama people admit Florida is hard to call, polls there taken in October show anything from Obama +8 to McCain +5. Indiana is just as weird, running from Obama +10 to McCain +7 in October polls. Iowa is like Colorado, an important state but with only two polls done there this month. Minnesota is strange as well, with a range of 18 points between reports from the eleven polls done there in October. Looking at Missouri, there have been nine polls done and they range from Obama +8 to McCain +3. Even in safe states there's some hinkiness, as New Jersey has a range of 15 points between polls taken in October. Like California, it's not in doubt but the volatile range of results is sending a signal about the polls' validity, just like the national polls. I don't think I need to go through all of the states to show what I'm saying here, go check out RCP and drill down to specific polls on specific states. The state polls are showing the same volatility that I noted in the national polls, and there's likely a common reason for it.

A reader mentioned Survey USA earlier this week, and I'd like to use them as an example of what I mean. First off, I like Survey USA for making internal data available; it really helps me take apart their process to see what they were thinking. And I found an interesting trend, something which is consistent with the national polls and which explains both the volatility and the invalidity of the current model.

2006 was a bad year for republicans, a year when republicans stayed home and democrats used the opportunity to win a number of close races and take over control of the House and Senate. In a number of states, therefore, it's not surprising that democratic party supporters gained a few points (usually 1 to 3 points) relative to 2004 in voter participation. So I went back and looked at voters by party affiliation, and compared those balances to this year's weighting by Survey USA. In thirty-six states, the party affiliation weights for democrats used by SUSA was five points or more higher than in 2006, a high-water mark for democrats. In twenty states, the party afiiliation weights for democrats used by SUSA was ten points or more higher than in 2006, and in eight states, the party affiliation weights used for democrats by SUSA was thirteen points or more higher than in 2006. Significant battleground states affected by this bias are as follows:

Pennsylvania: D+5 in 2006, SUSA using D+19, 15 point variance
Indiana: R+14 in 2006, SUSA using R+1, 13 point variance
Nevada: R+7 in 2006, SUSA using D+6, 13 point variance
Colorado: R+3 in 2006, SUSA using D+9, 12 point variance
Iowa: R+2 in 2006, SUSA using D+10, 12 point variance
Virginia: R+3 in 2006, SUSA using D+9, 12 point variance
Ohio: D+3 in 2006, SUSA using D+13, 10 point variance
Missouri: R+1 in 2006, SUSA using D+7, 8 point variance
North Carolina: R+1 in 2006, SUSA using D+5, 6 point variance

I've looked at the publicly available records on historical election participation, 2008 new voter registrations, and the Census information on these states, but I can find no valid reason for such large and arbitrary changes in political affiliation weightings. I would therefore submit that the models being used for many of the state polls have design flaws, which threaten the credibility of their published results.


TrackBack URL for this entry:

Listed below are links to weblogs that reference Cooking Polls - State Poll Edition:

» Conservatism Today linked with A Look at Polling Bias

» Pursuing Holiness linked with Wacky Conspiracy Theory o’ the Day

» Slublog linked with About Those State Polls...

Comments (36)

What you're missing is very... (Below threshold)

What you're missing is very simple: YOU ARE WRONG

from today:
"Barack Obama is up 11 points on John McCain among likely voters in the new Washington Post-ABC News tracking poll, 54 to 43 percent."

Since you're apparently a party loyalist, you have no interest in the amazing work from people like Nate at 538, nor Intrade, Pollster.com, or any other reputable polling source.

You're cherry picking polls, using very outdated polling data, and finding flaws where there aren't any. You're the polling equivalent to a WTC7 conspiracy theorist.

But, that's who you are and noone should be surprised that a party-loyalist such as yourself, would come to the conclusions you want to come to, using erroneous and mangled data.


ps: I've looked around a bit, and I can conclusively say "your blog sucks"

OK #1, you've convinced me ... (Below threshold)

OK #1, you've convinced me with your reasoned, impartial arguments. Well played.

DJ: I looked at the interna... (Below threshold)

DJ: I looked at the internals of two national polls today. Battleground and IBD. The interesting thing is that McCain is ahead among Hispanic/Latino and Catholics in both polls, which have had a big turnaround in the last week. If he wins those demographics, the election is over.

The weight of both groups in the polls makes a big difference. If IBD in over wighting Latinos and McCain is ahead that will bias the results.

Now what does makes sense in both polls is that among democrats and republicans the undecideds are small but over 20% of independents are undecided. Anyone can see the logic in that result.

#1 is just another Obama th... (Below threshold)

#1 is just another Obama thug, just like the thug in pittsburg who carved up the McCain supporter.

Thats going to play real well, black dude carves up white woman for voting McCain. Real American.

DJ-Thanks for doin... (Below threshold)


Thanks for doing this -- I suspect people are going to be very interested in these arguments, come November 5th.

After your last post, I was inspired to start looking at state polling data, and chose to look at North Carolina (a usually solid Republican state which state polls have been showing as in play). Not surprisingly, the polls were showing a dramatic and unprecedented shift towards Democrats in the Party ID numbers.

It turns out that there's been a massive increase in the number of REGISTERED Democrats in North Carolina -- large enough to potentially justify the seemingly crazy poll assumptions.

Is this a result of actual demographic shifts? ACORN's fraudulent registration efforts? Republicans and Independents switching to Democrats solely for the sake of voting in North Carolina's primary (May 6th, long after McCain had clinched the Republican nomination)? Some combination of the above?

No one appears to know the answer to these questions -- and without answers to those, there's no way to predict the vote results in North Carolina with any accuracy.

So... *shrug* I have no idea what's going to happen. But if the assumptions going into all these polls hold true across the country, the Republican Party has completely imploded.

WOW that 1st comment by the... (Below threshold)

WOW that 1st comment by the Obamatron really convinced me .........................................NOT !!

One way to tell that you ar... (Below threshold)

One way to tell that you are correct with any anaylsis is to watch how people who disagree respond.

Ohmmade is obviously thinking that Intrade is a responsible polling firm? Please!!! It is an open market system that is subject to short selling.

The blog, IF YOU READ IT, is about party ID. And the anaylsis DJ makes is spot on!

In 2004 the difference between party ID netted Dems +3...Yet a republican won the presidential election. How can you explain that?

Another thing Ohmmade that you are missing are two polls showing a 1 point lead to two polls showing a 10-14 point lead. How can you account for such a spread? How do you? You can't because this year there are two very important variables that have NEVER been tracked before.

1) First African-American nominee of a major political party

2) First female conservative VP nominee.

There has never been two energizing variables in any US presidential election. Therefore you can't use traditional methods to conduct a poll because the traditional models simply will not work. That is EXACTLY why Gallup is using a "traditional" and "expanded" method in their polling. They are experimenting with the "expanded" method to see which of the two models will make an accurate prediction with the two energizing varibles.

Since there are 14 polls that have a 14 point spread one can say we have a 14 margin of error. Which is not too accurate.

I see DJ looked into registrations, however he didnt give full details of Operation Chaos intiated by Rush. There were hundreds of thousands (if not millions) of Republicans who switched party id to democrat to vote in open primaries. This was done to keep Hillary in the battle. This increase in Dem registrations is being looked at by the pollsters and media and "excitment". On the other hand they are missing the fact of Operation Chaos. Of course pollsters must be taking this increase in registration (Party ID) and skewing party ID to a point that is not in a historical context of past elections. Now I ask how many of these Repubs switched back to Dems? Is that captured anywhere in any poll?

ANYTHING can happen. Also Obama's actions are not one of confidence. Rolling out Medicare statements is one of despiration. Plus a leaked internal poll showing a 2% lead in PA is very telling. He can't win the White (not a code word) House without PA and if Rendell is pushing the "all hands on deck" panic button and Hillary is coming out to help in a historically deep blue state...there are more serious issues. DJ has debunked SurveyUSA's recent poll given the lack of historical adherence to the +3 Dem party ID.

Finally Ohmmade in 2004, Kerry was in the lead by 10 points (avg) going into the final week, however Bush beat him. Obama is in the lead by 7 points (avg) going into the final week. It seems like history might just repeat itself given three polls have his lead today to under 3 points.

Look at IBD poll today. Th... (Below threshold)

Look at IBD poll today. The most accurate poll in 2004. It has a one point spread. so that obamabot might just want to take his hate somewhere else.

You were going great Rndguy... (Below threshold)
DJ Drummond:

You were going great Rndguy, until you said Kerry was leading late in 2004. Check


and you will see that just before the election, Marist had Kerry up by 1, Fox News had Kerry up by 2, CNN/USA/Gallup and ARG had it tied but everyone else had Bush ahead, from +1 (4 polls) to +6 (Newsweek Oct 29).

Here's the thing. SurveyUSA... (Below threshold)

Here's the thing. SurveyUSA famously doesn't weight to party identification at all.

So if 54% of the likely voters SurveyUSA gets in a particular poll say they're Democrats, that's what SurveyUSA reports.

In four polls in PA this cycle, their Democratic number has ranged between 50% and 54%; while they're not weighting to it, that figure is pretty close to what PA reports the registered voter breakdown is.

Maybe they're being screwed with by the respondents, maybe they're wrong in not trying to lock the number down to some external constant -- but maybe it's also the case that if you call 600 Pennsylvanians and ask them if they're Democrats or Republicans, and you do it four times in a row on four successive weeks, and it's always more than 50% dem, maybe it means something.

Thank you for the corretion... (Below threshold)

Thank you for the corretion! But wasn't Kerry ahead by 10 two or three weeks before the end?

Nope Rndguy, Kerry's last l... (Below threshold)
DJ Drummond:

Nope Rndguy, Kerry's last lead of more than 3 points was at the end of July.

You may be thinking, though, about the exit polls that were released early election day. Kerry's campaign leaked exit polls showing him with landslide leads in every early state. Later it turned out the guys doing the polling were all untrained college kids.

Oh, that reminds me Ken - p... (Below threshold)
DJ Drummond:

Oh, that reminds me Ken - pay attention to the in-state regional breakdown of polling. Philly and Pitt have very different numbers this year, and West Pa. has not forgotten Murtha or Obama's "bitter" remark. But the polls have focused heavily on metro areas.

I'm not a Mccain supporter,... (Below threshold)

I'm not a Mccain supporter, but I find your blog interesting to read nevertheless. I am not here to argue with you, math is not my strong point, so I leave that to you. But I wanted to point out that those who are referencing the IBD poll as being the most accurate might want to check out the internals... they have McCain beating Obama by over 50 points in the age 18-24 range. Sorry, there's something seriously askew with that, even if one thinks McCain has more support than the polls are indicating.

DJ-I believe you a... (Below threshold)
Roy Lofquist Author Profile Page:


I believe you are correct about not believing the polls. I have been watching elections since 1952 and have never seen such screwy results. Two points:

1. 2006 was NOT a blowout.

"In the preceding 6 midterm elections where the incumbent President's party lost seats the average loss in the Senate was 6.1, in the House 29.33. In 2006 the Republicans lost 7 in the Senate and 30 in the House."


2. Either the national polls are wrong or the state polls are wrong.

California, New York and Illinois are very strong for Obama (+15%). They comprise a large fraction of the voting population. If, as some of the more reliable polls indicate, Obama is up by 4 or 5% then McCain must be leading in many of the smaller States - contra the State polls.

Party allegiance has always been very strong in the US. The shift to the Democrats posited by the pollsters has but one precedent - the Old Confederacy. This was not because of race - as many would have you believe - but because of the anti-war takeover of the Democratic Party in 1968. There is nothing approaching this change of circumstances in the last 20 years.


rndguy...I'm a mccai... (Below threshold)

I'm a mccain supporter...I agree with DJ...but your comment:
Finally Ohmmade in 2004, Kerry was in the lead by 10 points (avg) going into the final week, however Bush beat him.
is incorrect...
Check out RCP average 2004 (It's available online) In fact bush had a narrow 1-2% lead almost always POST convention. It was always close..I'm talking about averages.
So...I don't think that means 2008 will be similiar. I think there are too many unpredented factors and for example in 2004 you didn't have two polls with 1 point and two others with 13/14. So we are in uncharted waters...no doubt!

Sorry...I didn't read far e... (Below threshold)

Sorry...I didn't read far enough. DJ got the point out already.

Thanks for the analysis wit... (Below threshold)
andre lee:

Thanks for the analysis without the hubris.

Check out this plot showing... (Below threshold)
Jeff C.:

Check out this plot showing polling results from the 2006 congressional election. Leading up to the election, every single poll overstated the magnitude of the Democrat victory. The actual margin was 7.8%. The polls ranged from 10% to 22%. The worst offenders were exactly who you would suspect, Newsweek, CNN and NY Times.


Take heart, despite what the Dems and their MSM cheerleaders would like you to think, it is not over.

Always striking that DJ nev... (Below threshold)

Always striking that DJ never considers polls for congressional races for confirmation of any of his hypotheses.

Republicans, question for you:

How many seats will you lose in the Senate, and

how many more will you lose in the House?

Do you think the good voters of, say, Colorado, who are in the process of electing a new Democratic Senator will "split the ticket" and still vote McCain???? FOR ONCE IN YOUR LIVES, GET REAL!!!

Haven't you Republicans inflicted enough damage already upon the nation and world -- do you truly think rational Americans want another four more years of this shit????

There's a reason for that, ... (Below threshold)
DJ Drummond:

There's a reason for that, Herman. People often vote differently for the President than for their Congressman.

Consider the Congresses Reagan and Clinton had to work with, for example.

I have thought about the congressional races, you are quite wrong there. I checked into them, and found that there is often no statistical correlation, much less causality, between a vote for President and the vote for the congressman. Different jobs with different scopes of authority, you see.

I have History on my side, sir. And you, as usual, have your opinion and not much else that I can see to back it up.

DJ,He also has cur... (Below threshold)


He also has cursing, capitalization and bold text on his side.

I'm not a Republican sir, I... (Below threshold)

I'm not a Republican sir, I'm a conservative.

Hestrold, if you want conse... (Below threshold)
DJ Drummond:

Hestrold, if you want conservative values to have a prayer of continuing, you'd better vote for the republican.

DJ -I'm wondering ... (Below threshold)

DJ -

I'm wondering how much, if any, the flood of registrations by ACORN and such skews the weighing. Is there any effect, or is it effectively discounted?

Gentlemen polling is part s... (Below threshold)

Gentlemen polling is part science and part art. Pollsters use statistical models to build a platform. From that platform they interview poll respondents that adhere to their model. What is known about any given area is the ratio of Republican,Democrat and independent voters. What is also known is the demographics of the voters.Poll results can vary according to the percentages that are used in the demographics.( The IBD poll out today has the 18-25 age group going to McCain by 74%)
To summarily dismiss poll results because they don't fit your ideology would be like somebody not readjusting their 401K portfolio because they don't want to acknowledge the losses in their portfolio.(Just throw the statements in a drawer) Polls are a tool to be used by the media and the candidates.The candidates are the ones who will show you what their pollsters are saying by where they go to campaign.National polls at this stage are nothing but statistical noise, more like a beauty pageant where the winner gets a pretty sash.Popular vote doesn't count except in each state and that's where the action is.
Right now McCain does not appear to have held the Bush states from 04. McCain has to play defense in VA, IN,MO,NC,FL,OH,NV and maybe even ND,GA and MT. If anybody suggested this scenario at the beginning of this election they would have been laughed out of the tv studio.If McCain needs to flip a Kerry state like PA or several states like WI,IA and MN, watch where his team campaigns. The important number is 270 not what the polls say.
The important question to ask if he losses the election is what happens to the Republican party? The answer to that will be determined by the autopsy that will follow,hopefully on the corpse of a dead campaign, not a still thrashing and twitching body.

dmacets-I'm sorry,... (Below threshold)


I'm sorry, but if you actually think Montana is in play, you haven't been paying attention.

In 2004, 39% of the voters were Republicans and 32% were Democrats.

In 2006, the party breakdown was exactly the same: 39% R, 32% D.

In the 2008 polls showing McCain with a low single-digit margin get that result by assuming completely ahistorical turnout results. ARG, for example, gets a McCain victory by 5% after assuming that 35% of the voters will be Republicans and 37% will be Democrats --- a huge, huge shift. What is the shift in estimated turnout based on?

If you use ARG's own published data, but assume identical party ID to that of the last two elections, then McCain is actually up by 11%.

If it's really true, as many pollsters appear to be assuming, that about one in every five Republicans have suddenly "seen the light" and become Democrats, sometime in the last two years, then it's true that all those states look like the polls say, and Obama is going to win in a landslide.

But I'd love to hear some actual evidence or argument supporting the notion that more than 15% of Republicans have become Democrats (or, equivalently, that 30% of those who voted in the last two elections will stay home this year). That's just an astonishing thing to be assuming, and should be *itself* front page news in every paper in the country. If there's any reason to believe it's true, that is.

Where did you find or what ... (Below threshold)

Where did you find or what information could you not find? I recomend checking out the The United States Elections Project because it, in some cases, does justifying large swings is party ID #'s.

For example, in Nevada there have been 257,000 new registrants, 49.7% were democrat, 23.2% were Republican, 20.5% were non-partisan, and the remaining balance is spread amongst 3rd parties. The situation is similar in NC as well.

To be perfectly honest I do not have the statistical knowledge (or data) to transform that into reasonable party ID weights. However, these #'s do portray a large disparity in registrations in two of the states you mentioned. Based on that, SUSA's party ID weighting may be justified. Your argument would be stronger if you would delve into what you think is a reasonable party ID estimate (and how you got there).

dmacets, I would like to re... (Below threshold)
Don S:

dmacets, I would like to respond to your statement: "Right now McCain does not appear to have held the Bush states from 04. McCain has to play defense in VA, IN,MO,NC,FL,OH,NV and maybe even ND,GA and MT. If anybody suggested this scenario at the beginning of this election they would have been laughed out of the tv studio". I don't think it is so strange that some of the southern states (VA, NC, GA) are apparently polling closer this year than usual. They have very high African-American populations, and there is no question that is a motivated demographic this year. The question is whether, ultimately, an increased share of African-American voters will be sufficient to actually flip one or more of those states to the Democratic candidate. On the other hand, some of the traditionally Democratic states, like PA, may be more prone to flipping Republican this year, because of relatively conservative Catholic Democrats who are unwilling to vote for a very liberal Democratic candidate, and who are attracted to a relatively moderate Republican war veteran who is paired with a female Vice-Presidential candidate. It is becoming clear that we are likely to see an unusual alignment of states this year because of these "firsts" running in both parties.

ORyan, you may be correct a... (Below threshold)

ORyan, you may be correct about the numbers of new registrations, but this tells us nothing about the actual vote, given ACORN's fraudulent registrations, which have been heavy in NV, OH, IN and other states currently thought to be toss ups. (The number I heard was 14 states, and in some cases there were more people registered to vote than were eligible to vote.) So if ACORN is only pulling registration fraud, the demographics don't shift as much as the registrations indicate. If they are pulling voting fraud as well, it might be a different story.

There are also the PUMAs. T... (Below threshold)

There are also the PUMAs. These are the people who voted for Hillary but won't vote for Obama. I've heard estimates of 10% to 40% of the 18 million will be pulling the lever for McCain.

In military terms, this whole situation is "fluid."

DJ,Do you suppose ... (Below threshold)


Do you suppose that on the morning of election day we will hear that Obama has already won the election thanks to "exit polls"?

I'm pretty sure this will be the final effort of the Obama campaign at NBC, CBS, ABC, CNN, MSNBC to try and win the election.

First you build the narrative, that Obama is so far ahead it's already over. Then you confirm the narrative as early as possible, thus creating reality.

How do we who want folks to get out and vote on election day across the U.S. get them out to vote when almost for sure something like this is going to happen to keep people at home?

Just like the Florida pan handle in 2000.

I think it WILL play exactl... (Below threshold)

I think it WILL play exactly like the woman who claimed to have been assaulted by tall black man at her ATM.


she has confessed to making up the entire story


I'm with Hestrold on this. ... (Below threshold)
Laszlo Toth, Jr.:

I'm with Hestrold on this. I can see how one can be a conservative. I can see how one can be a Republican. But ever since Republicans have gotten into this whole, "My party, right or wrong, and the facts and the country be damned," attitude that arrived with George W. Bush, I don't see how one can be both.

I was a McCain supporter in 2000. To see him throw away almost every single policy of that campaign, to please the extremely narrow "base" -- minor things like duty, honor, country, secular government, obedience to the constitution, reasonable fiscal policy, force as a last resort, small things like that -- has been heartbreaking.

In almost every respect worth talking about, the Democrats have become more conservative than the Republicans. Reagan, Goldwater, and Eisenhower wouldn't recognize today's party -- and probably wouldn't be allowed in.

Reagan's questions from 1980 are just as applicable today as then: "It might be well if you ask yourself are you better off than you were four years ago? Is it easier for you to go and buy things in the stores than it was four years ago? Is there more or less unemployment in the country than there was four years ago? Is America as respected throughout the world as it was? Do you feel that our security is as safe? That we're as strong as we were four years ago? And if you answer all of those questions yes, why then I think your choice is very obvious as to who you'll vote for. If you don't agree, if you don't think that this course that we've been on for the last four years is what you would like to see us follow for the next four, then I could suggest another choice that you have."

McCain is right: Americans are being asked to make a risky choice, on someone untested, flighty, and prone to anger at the drop of a hat.

Fortunately, Barack Obama is available as a conservative alternative to that un-American radicalism.

And the fact that Hestrold and I clearly exist, in a thread all about skepticism over whether the Republican Party has drifted so much into the far right as to kick Reagan conservatives such as myself out...

Well, I think it speaks volumes about who's making bad assumptions as to what's happening in the real world. And it ain't on the part of the pollsters.

Polls are stupid and any on... (Below threshold)
Mr Bill :o:

Polls are stupid and any one using polls to justify anything is lazy and dishonest
That is all.

It don't matter if you a co... (Below threshold)

It don't matter if you a conservative, republican, or even a moderate when the clear choice is mccain and lower taxes to obama and higher taxes. polls are always heavy leans toward democrats and this year seems to be even more so. too bad the results are less favorable toward democrats (cough) if they lose they will be so disapointed.






Follow Wizbang

Follow Wizbang on FacebookFollow Wizbang on TwitterSubscribe to Wizbang feedWizbang Mobile


Send e-mail tips to us:

[email protected]

Fresh Links


Section Editor: Maggie Whitton

Editors: Jay Tea, Lorie Byrd, Kim Priestap, DJ Drummond, Michael Laprarie, Baron Von Ottomatic, Shawn Mallow, Rick, Dan Karipides, Michael Avitablile, Charlie Quidnunc, Steve Schippert

Emeritus: Paul, Mary Katherine Ham, Jim Addison, Alexander K. McClure, Cassy Fiano, Bill Jempty, John Stansbury, Rob Port

In Memorium: HughS

All original content copyright © 2003-2010 by Wizbang®, LLC. All rights reserved. Wizbang® is a registered service mark.

Powered by Movable Type Pro 4.361

Hosting by ServInt

Ratings on this site are powered by the Ajax Ratings Pro plugin for Movable Type.

Search on this site is powered by the FastSearch plugin for Movable Type.

Blogrolls on this site are powered by the MT-Blogroll.

Temporary site design is based on Cutline and Cutline for MT. Graphics by Apothegm Designs.

Author Login

Terms Of Service

DCMA Compliance Notice

Privacy Policy