« A Boy Named Hsu & Small Donors Concealment | Main | Obamanomics Tutorial: By Aaron, Age 33 »

Odds at Ends - The Pew and Battleground Polls, with a Gallup Chaser

I've laid out a pretty harsh accusation against the polls this year, by claiming that all the major polls are far from accurate. The cause of this, in essence, has been that the polls made some key assumptions about turnout, the independents, and the undecided voters. Assumptions which they never tested, and now are finding cannot be trusted. Poll results vary wildly from one another, and not just at different times. The variance for current polls listed at Real Clear Politics for this morning ranges from the Pew poll which advertises a 14-point lead for Obama, to the Battleground poll which says the lead is only 2 points. The variance is too great (and there are polls relatively close to both ends, demonstrating proof of statistical invalidity for the published confidence level) for even the casual observer to accept as a reasonable. There are four polls which show a 10 point lead or greater for Obama, and another five which show a 6 point lead or less. It is mathematically impossible for so many polls to be valid, yet disagree to such a degree with valid methodology. I said this when McCain was ahead, again when Obama climbed in front, and I am repeating it yet again. The starting point to discussing the polls this year, is understanding that the methodology in common use is flawed, and is producing results which cannot be depended upon.

A quick word here about validity. Opinion polling relies on statistical math, which depends on certain key tests. When a group of respondents exceeds a certain size, a pattern of responses emerges which is generally symmetrical, with few outliers. A p-test can be done to confirm that the results are consistent with the requisite conditions. This produces what is known as a confidence level, which in common words means the likelihood that the process, if repeated using samples from the same data pool and using the same method, will result in the same conclusion. This is called reproducibility, and it is the most signifcant test for human behavior testing. The confidence level basically sets out how often the same results should be expected to repeat. The most common published level of confidence in opinion polling is 95%, which predicts that the same method used at the same time will produce results within the published margin of error no less than 19 out of 20 times. RCP is listing twelve major polls with variances from each other which cannot be covered by the MOE, which proves the model is invalid by its own definition. One poll as an outlier could be explained, but the range is too great to explain the variance between the rest.

People have specifically asked me about the Pew and Battleground polls, since their twelve-point variance is the largest. I will have to say that in both cases, the error has been the same - disregarding historical norms in favor of introducing subjectively chosen demographics, things like over-sampling urban areas, younger voters, and democrats which creates a false image relative to the voting demographic. But to get a sense of the numbers, I would like to examine the Pew, Battleground, and Gallup polls in the context of their direct movement, and in reweighting their party affiliation to historic norms.

First, here are the recent results from Pew:

Sept 17 - 46% Obama, 44% McCain, 10% undecided
Sept 30 - 49% Obama, 42% McCain, 9% undecided
Oct 13 - 50% Obama, 40% McCain, 10% undecided
Oct 20 - 52% Obama, 38% McCain, 10% undecided

Pew is showing what is effectively a zero-sum game, with Obama gaining directly at McCain's expense, with around 10 percent remaining unsure each time.

Now, the Battleground trend:

Sept 25 - 45% Obama, 47% McCain, 8% undecided
Oct 3 - 45% Obama, 41% McCain, 14% undecided
Oct 9 - 48% Obama, 38% McCain, 14% undecided
Oct 16 - 47% Obama, 40% McCain, 13% undecided
Oct 22 - 49% Obama, 47% McCain, 4% undecided

Battleground shows support movement more independent between the two candidates, and the most recent undecided numbers are much lower than what we saw before.

- continued -


OK, with that in mind, let's check the Gallup numbers for the same range of dates:

Gallup Daily Tracking
Sept 14: Obama 47%, McCain 45%, 8% undecided
Sept 21: Obama 48%, McCain 44%, 8% undecided
Sept 28: Obama 50%. McCain 42%, 8% undecided
Oct 5: Obama 50%, McCain 42%, 8% undecided
Oct 12: Obama 51%, McCain 41%, 8% undecided
Oct 19: Obama 52%, McCain 41%, 7% undecided

This model appears to be similar to Pew's, zero-sum balancing with a constant undcided portion.

Gallup 'expanded voter'
Oct 8: Obama 52%, McCain 43%, 5% undecided
Oct 15: Obama 51%, McCain 45%, 4% undecided
Oct 21: Obama 52%, McCain 42%, 6% undecided

This model allows a smaller undecided portion, suggesting that undecideds are pressed for a clear decision. Also, Gallup has admitted that this model has no precedent, and uses over-samples of urban and youth voters, in the presumption that they will sharply increase participation this year.


Gallup 'traditional'
Oct 8: Obama 50%, McCain 45%, 5% undecided
Oct 15: Obama 49%, McCain 47%, 4% undecided
Oct 21: Obama 51%, McCain 44%, 5% undecided

This model removes the urban and youth voter overweights, but otherwise is the same as the 'expanded voter' model. This is because Gallup abandoned the true historical model, and so can only attempt to recreate it to some degree by using data sets it knows have been corrupted by invalid methodology.

Now, let's see what happens to these results when the internal data is reweighted to historical party affiliation norms:

Pew:
Sept 17: 46-44 Obama becomes 45-46 McCain
Sept 30: 49-42 Obama becomes 48-44 Obama
Oct 13: 50-40 Obama becomes 48-43 Obama
Oct 20: 52-38 Obama becomes 50-42 Obama

Bear in mind that this accepts Pew's polling methodology, which may have over-sampled other demographics besides just democrats. For example, in the Oct 20 poll Pew undersamples seniors and oversamples the 50-64 age group, oversamples high school only education by a large amount, and fails to note regional breakdowns or the urban/suburban/rural split. These are critical points which Pew fails to address, and which hshould make the reader wary.


Battleground:
Sept 25: 45-47 McCain becomes 44-47 McCain
Oct 3: 45-41 Obama becomes 46-44 Obama
Oct 9: 48-38 Obama becomes 48-41 Obama
Oct 16: 47-40 Obama becomes 49-42 Obama
Oct 22: 49-47 Obama (no internal data to reweight yet)

Battleground does not change much, but this reinforces that something odd has happened in the recent poll, most likely among independents.


Next to check is the reweight of Gallup's polling:

Gallup Daily Tracking (d=daily, e=expanded, t=traditional)
Sept 14: 47-45 Obama (d) becomes 43-44 McCain
Sept 21: 48-44 Obama (d) becomes 43-42 Obama
Sept 28: 50-42 Obama (d) becomes 43-42 Obama
Oct 5: 50-42 Obama (d) becomes 44-41 Obama
Oct 12: 51-41 (d), 53-43 (e), 51-44 (t) Obama becomes 47-39 Obama
Oct 19: 52-41 (d), 52-42 (e), 51-44 (t) Obama becomes 45-42 Obama

Note that the reweights of these polls using historical norms are much more consistent with each other.

The next thing I suggest is looking at comparable metrics. First, base party support:

Support for Obama by Democrats
Pew: 87% 92% 91% 91%
Battleground: 82% 81% 83% 86%
Gallup: 86% 86% 86% 87% 88%

Support for McCain by Republicans
Pew: 90% 86% 91% 89%
Battleground: 85% 82% 80% 81%
Gallup: 84% 84% 82% 82% 84%

Note that Pew reports the highest support within each party. Note also that McCain's support by republicans is reported to be dropping.

Next, independent support:

Independents

Pew: (for Obama) 38% 38% 45% 51%
Pew: (for McCain) 45% 46% 37% 33%

Battleground: (for Obama) 32% 38% 41% 43%
Battleground: (for McCain) 40% 36% 34% 33%

Gallup: (for Obama) 22% 22% 23% 33% 27%
Gallup: (for McCain) 31% 31% 32% 25% 34%

Note that Pew reports a commanding lead for Obama among independents, while Gallup shows McCain in consistent advantage.


Undecideds

Pew: 08% 07% 08% 07%
Battleground: 15% 15% 15% 14%
Gallup: 16% 16% 15% 14% 14%

Battleground and Gallup agree that there is a lot of the population still waiting to be won over. The election is certainly well within any reasonable boundaries of doubt.


Conclusion

It's difficult to work with limited internal data, especially when the polling group has altered more than one category of demographic. But it is interesting to note that when the reported data is reweighted to a consistent historical norm, that even the varied results from three different polling groups start to trend in the same way. Obama supporters can take comfort from the indications that his lead stands up to inspection, albeit not as large, while McCain supporters can take heart in the evidence that the election is not at all decided, that the level of turnout, the choice of the independents, and which way the undecideds break (including the choice to stay home) is a vital part of the decision still to be resolved.


Pew's data comes from here

Battleground data comes from here

Gallup's polling reports come from here

Gallup's internal affiliation numbers are reported here


TrackBack

TrackBack URL for this entry:
/cgi-bin/mt-tb.cgi/32438.

Comments (27)

I said yesterday in a post ... (Below threshold)
Rndguy:

I said yesterday in a post here and I say it again today. The pollsters are "trying" to get this "right" because there is no historical precident for two very different factors in this election--1) First African-American nominee 2) A conservative female VP nominee.

That is why the polls vary today from tie to 14 points. There is no way to put these factors into perspective with a traditional polling model. Gallup is "trying" an experiment to see how it all turns out.

When I read the Pew and Battleground internals I found it interesting that they do not poll much of where I live and is called by many as "fly over country". They mostly poll in the Northeast, South and West coast.

There is a silent majority that really is getting ticked off on the daily montra from the media that 'It is over, don't bother showing up'. Please see Drudge's headline today. See CNN's statement on McCain giving up on CO..when Palin is breaking attendence records. The media may be too far into the tank that it may and most likely will backfire into McCain support...Well I digress...back to my topic...

PA is in play more than what we are being told. Again look at the leaked internal poll yesterday from Obama's camp. They have never denied the 2% lead they have. To back up the dire situation of the 2% PA lead is to look at Rendell pushing the "all hands on deck" panic button to get Obama and the Clinton's back into his state b/c McCain has inspired conservatives (along with Murtha, but I want to stay on topic).

Also another thing that is not in the polls is Rush's "Operation Chaos" during the primaries. I know there were a HUGE number of Repub's who registered Dem to vote for Hillary. This is being forgotten. Remember the huge numbers of "new" democrats declared by the media and campaigns?

Given some have changed their party affiliation back, however I don't know if this was measured? I do know the Ohio Sec. of State was not happy because of the increase of new registrations caused by Rush..I bet a majority of Repubs who were involved in this have not swithced back and could be adding to the huge increase in the Dem party affiliation.

Simply put, WAY to MANY variables that are not in any pollster's models are out there. Very difficult to get a handle. The key to watch is "where is the canidate today?". Where are they? They are in states where internals show they have a chance.

If Obama looses PA it is all over for him, he can't win even if McCain gives in CO, NM and IA. PA is the battleground and Obama's poll confirms it is in "play" as stated by Wiz today.

Are you sure BG primarily p... (Below threshold)
Good Captain:

Are you sure BG primarily poles on the coasts and in the south? I would assume (w/ their approach of state by state that would not be the case. Then again, assumptions kill.

I live in the "Central Plai... (Below threshold)

I live in the "Central Plains". I define that as IA (7), MN (10), SD (3), ND (3), WI (10), NE (5), MO (11), IL (21) and KS (6). Total 95 Electoral votes or 35% of the total to win.

The sample shows that 78 of 1011 from the "Central Plains" were surveyed or 7% of the sample. I think that is out of balance. Given that most of the Central plains is mostly red, with MN, WI, IL being blue and IA (where I live) kind of purple.

So..with 35% of the EV total, the "Central Plains" were only 7% of the sample. This goes to the point EJ made. They just aren't getting it right. I am not a statistical genius, but I can add, multiply and divide fairly well.

<a href="http://www.tarranc... (Below threshold)
Rndguy:

http://www.tarrance.com/files/GWU-BG-Public-tables-10.16.pdf

Battleground table for reference. And correction that should be DJ not EJ...Sorry...

Thanks Rndguy! The % for C... (Below threshold)
Good Captain:

Thanks Rndguy! The % for Central plains does seem a bit low. The next question then would be how this (or these %'s) mirror past elections' voting %'s? If there were significant discrpancies, that might indicate a problem.

I've now seen DJ's ideas he... (Below threshold)
Parthenon:

I've now seen DJ's ideas here linked to on Oliver Willis, Instapundit and Instaputz. Impressive little collection.

Absolutely excellent analys... (Below threshold)
Therese:

Absolutely excellent analysis DJ Drummond! Thank you. Also, for all of you adding comments with more statistical insight, thank you very much too!

Obviously, this election is not over and McCain supporter turnout is going to be crucial.

This also reaffirms to me that the Obama camp and media are trying to use "psychological warfare" to win the election. As such, I think that they are trying to do two things:

1. Demoralize and discourage McCain/Palin voters from even showing up at the polls. And further get it so that these voters don't talk about McCain/Palin and continue to sell other voters on them.

2. Use a "go with the winning crowd" tactic to encourage Undecideds to vote Obama.

Therese...I think you are s... (Below threshold)
Rndguy:

Therese...I think you are spot on with your anaylsis of the motivation behind the propaganda we are hearing from the media.

One problem...The country has always been center right....like I said in my post above, the media drumbeat will only server to backfire and Obama will be the victim.

The country has always b... (Below threshold)
Parthenon:

The country has always been center right....

Prove it.

DJ,One question ab... (Below threshold)
Jeff:

DJ,

One question about party affiliation and the battleground states. Given that the pollsters aare not using historical PA %'s and given that the National PA % numbers would include deep blue and deep red states isn't it likely that the historic PA %'s for the battleground states would be even closer than the 38-34 historical national average (aka 4 point spread).

Given the pollsters are using an 8,10, or 12 point spread to evaluate their battleground polls aren't they completely off the charts here ?

Seems to me that the battleground states would have a lower than national average PA split.

So far, for any poll that I've seen the internals on it is clear that a 4 point PA Dem advantage shows Obama losing. They have to skew the PA %'s to 8, 10 or 12 to give him his leads.

I would love to see what th... (Below threshold)

I would love to see what the polls would look like if the media was appropriately covering the news of a presidential candidate who surrounds himself with racists, slumlords, socialists, communists, and unapologetic domestic terrorists. Obama would be polling at about 15% right now. I look at The Friends and Associates of Barack Obama and it doesn't make any sense that I have to go here to piece it together with links from all the mainstream sources (which acknowledge individual facts but never the whole picture).

Parthenon--Why must I prove... (Below threshold)
Rndguy:

Parthenon--Why must I prove it to you. Why can't you prove me wrong?

Why do you think every democrat presidential canidate moves to the center? Because they know they can't get elected with their liberal views.

Most successful Example: Bill Clinton

Obama is trying it, but his "spread the wealth" statement has exposed him as a liberal to the extreme.

FOX news is reporting that ... (Below threshold)
Necromancer Author Profile Page:

FOX news is reporting that bho has a 9 point advantage.

Well for one thing, Rndguy,... (Below threshold)
Parthenon:

Well for one thing, Rndguy, you made the assertion. It's your job to provide evidence for your assertion, not my job to disprove it. But what the heck.

In answer to your question, the word 'liberal' has been very effectively demonized, and in any case all presidential candidates move to the center after the primaries. But why should we stop there? (I chose these polls randomly, selected for short questions rather than any sort of cherry-picked liberal bias) All numbers are percentages, obviously. Via pollingreport.com...
---------------
"Which of these do you think is more important: providing health care coverage for all Americans, even if it means raising taxes, OR, holding down taxes, even if it means some Americans do not have health care coverage?"

Coverage for all:66
Holding down taxes:31

"Do you think it's the government's responsibility to make sure that everyone in the United States has adequate health care, or don't you think so?"

Think it is:61
Don't think so:35

--------------------

"Should gay and lesbian couples be allowed to marry, giving them full legal rights of married couples, or not?"

Should:47
Should not:47

---------------------

"All in all, which is more important: conducting stem cell research that might result in new medical cures, or not destroying the potential life of human embryos involved in this research?"

Stem cell research:51
Not destroying potential life:35

-------------------

"As I read off some different groups, please tell me if you think they are paying their fair share in federal taxes, paying too much, or paying too little. How about [see below]?"

lower income people
Fair share 32
Too little 13
Too much 51

Upper-income people
Fair share 24
Too little 63
Too much 9

Corporations
Fair share 15
Too little 6
Too much 73
--------------------

Alright, lot to swallow there, sorry about that. I couldn't think of a better way to make my point. Now where's your evidence?

"the word 'liberal' has bee... (Below threshold)
Whitehall:

"the word 'liberal' has been very effectively demonized"

They did it to themselves. Great Society, AFDC, the Warren Court, the whole Carter Administration, etc, etc, etc.

One big "unknown unknown" are those who refuse to participate in polls. I know many people who don't want to get involved with polling as they perceive them to be political tools rather than fact-finding efforts.

If ( I said if) in fact the... (Below threshold)
Les Nessman:

If ( I said if) in fact the media are using flawed polls then they are playing a dangerous game by abdicating their journalistic responsibility. If the polls show Obama leading by large margins all this time but he actually ends up losing the election, there will be a lot of outraged supporters and a divided Republic. More so in this particular election.

The questions about the polls are mostly ignored except by people like DJ. It seems to me that these are serious questions that need more people looking into them.

While your politics might b... (Below threshold)
hyperbolist:

While your politics might be the opposite of mine, DJ, you've clearly put in a hell of a lot of time and effort into your analyses these past few months. I'm sure you'll do fine with accountancy, but with the body of work you've put together here, it'd be surprising if a news organization or advocacy group of some kind doesn't offer you work in polling analysis. Ever consider it?

Parthenon----to play into y... (Below threshold)
Rndguy:

Parthenon----to play into your percentages, what is the weighting done on your polls?

Also Parthenon...an example of center right nation is to look at party ID from 2004. Which, DJ, has pointed out in his brilliant anaylsis in this blog.

IF party ID is +3 Dems for the last two election cycles, why is it the Republican has been elected? If the party ID was +3 dems the Dems should have won the last TWO elections, which of course we know the didn't. That is my proof this country is center right...

This year will be no different. The country is no different today...Party ID is way out of whack because of Operation Chaos.....Ohio Sec. Of state was going press charges on Rush b/c of this because he was inciting fraud (which he wasn't b/c Ohio is an open primary).

Where is that indicated in any polls? Answer it is not..so we must go back to that last untainted party ID which is 2004...Dems +3 yet a republican won by +3. GO figure...

Look at 10/22/08 IBD-Tipp p... (Below threshold)
I-Man:

Look at 10/22/08 IBD-Tipp poll. 53 % of 18-24 are voting for McCain - on 10/20/08 it was 44%. These polls just don't get past the smell test just as DJ has been telling us.

The poll that will matter m... (Below threshold)
Herman:

The poll that will matter most will take place on November 4th, and, I'm warning you, conservatives, if you try to do something like what happened to this poor guy here:

http://io9.com/5058447/homer-votes-for-obamapisses-off-president-mccains-death-bots

that is to say, if you again try to diebold the election, THERE WILL BE RIOTING IN THE STREETS. Indeed, at the Democratic Underground today there was a discussion as to how this rioting would occur (to be sure, some favoring nonviolent protests).

At this time four years ago... (Below threshold)
MPR:

At this time four years ago Kerry was ahead by 10 points across the polls. It looked as though he would sweep Bush out of office. The war was the issue and we were bogged down and no end in sight. But, the economy was still growing. In fact for the last eight years there hasn't been a time when we had two consecutive quarters of negative growth which is the definition of a recession. We are not in a recession. There are some parts of the country that are down right depressed because some industries are in a localized declining cycle.
Kerry is a liberal but, Obambi makes him look like a conservative. Veterans from a past war made the case that Kerry could not be trusted to take over the present conflict. Even though he had participated in and is a veteran of the past one. It was the war, the war, the war.
He lost Ohio narrowly and the election but, he lost in just a few other states narrowly also.
There was just enough doubt about Kerry to turn the tide.
There is still enough doubt out there about Obambi. His base is solidly behind him but, he is not closing the deal with Independents or "those" in the middle and I don't think he can. There is just too much unknown about Obamba and those in the "middle" are too stubborn to be talked into anything.
McCain has been in the public eye, warts and all, for a long time. I have thrown shoes at the TV at times because of his stand on some issues. He has tried to work with other side as his record shows but, gets stabbed in the back again and again. Shame on them, not him.
And then there are those that actually make their mind up in the polling booth.
After awhile the polls and the numbers just blur together and the 24 hour news cycle can make you lose sleep. My vote will not count for the Presidency because I live in a triple blue state. There are local Reps. and issues that my vote will make a difference.
I won't be staying home.

MPR---the rest of us conser... (Below threshold)
Rndguy:

MPR---the rest of us conservatives appreciate your sacrafice in the middle of such deep blueness. Thank you for carrying on the battle!

Parthenon:Did you ... (Below threshold)
Larry:

Parthenon:

Did you get this right?

Corporations
Fair share 15
Too little 6
Too much 73

????????????

If so, I am amazed that 73% got it right.

22. Posted by Rndguy said:<... (Below threshold)
MPR:

22. Posted by Rndguy said:
MPR---the rest of us conservatives appreciate your sacrafice in the middle of such deep blueness. Thank you for carrying on the battle!


Thank you. Rndguy
"Qui tacet consentit"!

DJ is right; guys this thin... (Below threshold)
Pam:

DJ is right; guys this thing is not over. We need to Turn out, Turn out, Turn out!

Larry, no I didn't - I flip... (Below threshold)
Parthenon:

Larry, no I didn't - I flipped them. 73 said too little, six said too much. The perils of hand transcription. Double checked the others; according to pollingreport.com, they are correct.

MPR wrote;"At this... (Below threshold)
Baggi:

MPR wrote;

"At this time four years ago Kerry was ahead by 10 points across the polls."

According to Real Clear Politics, this isn't the case at all. Bush lead in the polls almost the entire time, not by as much as Obama, but he still lead in the polls.




Advertisements









rightads.gif

beltwaybloggers.gif

insiderslogo.jpg

mba_blue.gif

Follow Wizbang

Follow Wizbang on FacebookFollow Wizbang on TwitterSubscribe to Wizbang feedWizbang Mobile

Contact

Send e-mail tips to us:

[email protected]

Fresh Links

Credits

Section Editor: Maggie Whitton

Editors: Jay Tea, Lorie Byrd, Kim Priestap, DJ Drummond, Michael Laprarie, Baron Von Ottomatic, Shawn Mallow, Rick, Dan Karipides, Michael Avitablile, Charlie Quidnunc, Steve Schippert

Emeritus: Paul, Mary Katherine Ham, Jim Addison, Alexander K. McClure, Cassy Fiano, Bill Jempty, John Stansbury, Rob Port

In Memorium: HughS

All original content copyright © 2003-2010 by Wizbang®, LLC. All rights reserved. Wizbang® is a registered service mark.

Powered by Movable Type Pro 4.361

Hosting by ServInt

Ratings on this site are powered by the Ajax Ratings Pro plugin for Movable Type.

Search on this site is powered by the FastSearch plugin for Movable Type.

Blogrolls on this site are powered by the MT-Blogroll.

Temporary site design is based on Cutline and Cutline for MT. Graphics by Apothegm Designs.

Author Login



Terms Of Service

DCMA Compliance Notice

Privacy Policy