« Gaza Phones Ringing American Voters For Obama | Main | More Obama Audio: Constitution Reflects Fundamental Flaw »

Gallup In The Tank?

Back in 2004, I jumped pretty hard on John Zogby. Zogby did two things which I considered, and still do, to be unacceptable conduct for a pollster. First, was that Zogby flat-out called the election for Kerry back in May of 2004, a prediction he hung onto through the rest of the campaign. The second reason was that Zogby started mixing results from his telephone polls with his online polls, which invalidates the results from both methods. I would also point out to the reader that in 2004 and 2005, I was unhappy with political affiliation weighting at the time, and had adjusted my own expectations by reversing the bias from polls. My point is that even four years ago I was challenging poll methodology when it deviated from NCPP guidelines, and even if Zogby is publishing prettier headlines now, that does not change my wariness from past experience. I will challenge any behavior at odds with valid practices.

This year, all of the major polls show Obama ahead in the presidential campaign right now, some saying he is well ahead. I found serious problems in their fundamental assumptions, not the least being the heavy weighting of democrats in the polls (and let's not mince words - any poll weights by party affiliation, the ones which simply accept what is called in are just accepting the raw data as demographically accurate, which is just as absurd in terms of party affiliation, as it would be if they assumed that race, gender, age, or educational demographics did not need to be reweighted). I have wondered two things as the campaign moved along - what would I say if I turned out to be completely wrong, and what would these polling groups say if I turned out to be right and they were the ones who blew it? For my case, I intend to review the election from a statistical standpoint, and if Obama wins in a landslide because the nation really did decide it was 48-25-27 DRI, then I will admit it plainly and take my lumps. I suspect the polling groups will have a harder time being forthright if my argument turns out to be correct. One reason for that is today's polling discussion from Gallup.

Gallup has noted the strength of early voting this year. The most significant points from that article are these; early voting is stronger than expected this year, and so far republicans have been just as eager to vote early as democrats. The third point is the most important signal of all. Says Gallup; "Early voting ranges from 14% of voters 55 and older (in aggregated data from Friday through Wednesday) to 5% of those under age 35. Plus, another 22% of voters aged 55 and up say they plan to vote early, meaning that by Election Day, over a third of voters in this older age group may already have cast their ballots."

The last two statements are very good news for McCain and bad news for Obama. This is because it demonstrates that enthusiasm to actually vote by republicans is equal to enthusiasm to vote by democrats. This runs directly against claims made in polling up to now, demonstrating that participation in polls is not directly related to voting this year. Second, the higher participation by senior voters and weaker participation by younger voters is directly in line with historical norms, again running against the poll expectations that this year would see a wave of young people voting but seniors staying at home. Gallup's own data proves this is not happening as they predicted, and the polls are therefore invalid in those respects, in addition to obvious flaws in the party weighting. The reasonable expectation from these facts, would be for Gallup to back down and correct its weighting to match the observed behavior. As of yet, Gallup has not taken that step. They did, I note, tacitly admit that the "expanded voter" model they introduced this year is invalid, but now they are running no less than three models of polling, which makes me wonder if they are going to wait to see which one comes out the closest (or the least embarrassing) and call that one their 'official' call - when a major polling group throws out three guesses instead of just one judgment, you can be sure they have lost confidence in their system.


TrackBack URL for this entry:

Listed below are links to weblogs that reference Gallup In The Tank?:

» Classical Values linked with The PUMA Question

Comments (30)

I think that, with this bei... (Below threshold)

I think that, with this being the last full week of campaigning before the election, that we are going to see the polls begin to tighten, as all of these polls who have been in the tank for Obama begin to become concerned about their own validity and future influence, start to show actual results, rather than skewing the data as they have been doing. In fact, I would not be surprised if a "major" poll did not get released later this week that shows McCain actually leading. The object lesson of all this should be to never trust the polling data, and to never let polling data influence your decision to vote.

DJI think that the... (Below threshold)


I think that the base of both parties are going to turn out heavily. We know that dem registration is way up and so the question is how many of the new registrants turn out to vote. If it's heavy, that's bad for McCain.

Here in Iowa Obama's ground organization is incredibly strong as I think it is around the country. I read some quotes from repubs in Florida about how strong his organization is there. Another bad sign for McCain.

Finally, I think you have to look at what the candidates are doing. It looks as if MCain is just battling to hold onto the red states which, again, is a bad sign for him.

Here's my (un)scientific po... (Below threshold)
Pretzel Logic:

Here's my (un)scientific poll: I drove through Wisconsin: Milwaukee, Green Bay and Madison areas the past 2 weeks. I was counting in my head, I saw many more McCain signs then I did Obama. There are a lot of both to be sure. Also, I got many honks and thumbs up for my NOBAMA bumper sticker.

Make sure everyone votes!

The ground organization for... (Below threshold)

The ground organization for Obama is indeed very strong. In fact it might be too strong. You can only contact someone to the point where any more contacts gets either the (false) answer you are looking for or you piss people off to the point where they aren't listening anymore.

Actual numbers for early tu... (Below threshold)
DJ Drummond:

Actual numbers for early turnout, JFO, are close to even for the two major parties from what I am hearing. The dem registration numbers are up, but they were up in 2004 as well and the new registrees then failed to show. Most of this years' new registrees signed up in the primaries, which means some of these will be pseudo-dems (Operation Chaos) and some will be PUMAs.

As to where the candidates are campaigning, Obama and McCain are spending time in both "red" and "blue" states. They've both invested in Colorado, but also Pennsylvania, in North Carolina but also New Hampshire. It seems that whatever polls the candidates see, are telling them a much different story than what the media-released polls are saying.

I noted what you said about 'hearing' from republicans in Florida, and I could tell you some very intriguing things I have heard from democrats in Philadelphia and Milwaukee. We'll have to wait and see how many of those anecdoctal stories hold up in the actual election.

One last thought; the 'early voting' numbers reported do not generally include absentee and overseas military voting, and I understand those are very heavy this year as well.

A number of my younger frie... (Below threshold)

A number of my younger friends are extremely pro-Obama. They talk about how great he is, and how he's going to win, et bloody cetera.

...and when I ask them if they're registered, about a third say "no," and change the subject. Of the rest, when I ask them where they're going to vote, they don't know.

Yeah, that's gonna work out just fine...

I've heard some anecdotal s... (Below threshold)
Randy R:

I've heard some anecdotal stories that the ground game has been closing as well, especially since Sarah Palin came onboard. She has brought in an influx of fresh energy and volunteers. Ultimately I think that Obama's alleged GOTV advantage will turn out to be overrated as well.

And DJ is right about McCain and Palin hitting PA and NH hard, but ultimately the fight was always going to be primarily in red states since Obama needs some of them to flip in order to win while McCain can win without a single blue state flipping.

Tell us now!... (Below threshold)
Pretzel Logic:

Tell us now!

One thing I think needs to ... (Below threshold)

One thing I think needs to be said is that the RCP average is useless this year; when one poll's mean is five or six sigmas from another one, taking the average produces a meaningless number, tantamount to saying "the election will be 55/41 and 48/47 AT THE SAME TIME."

Hi DJ,What are the... (Below threshold)
Joel Tamburo:

Hi DJ,

What are the historical DRI ratios? That will help in guessing how far off these polls are.

The polls are more difficul... (Below threshold)

The polls are more difficult to read this year than in other one. Essentially the RCP average is crap because we're, apparently, dealing with a bimodal distribution, where one group has an Obama landslide and the other has essentially a tossup race.

What I'm most interested in seeing his how the voter turnout gets affected by these "IT' OVER!!!" polls. If people ignore the polls and go vote, you might see some dumbfounded people on election night. If not, well, it could be an early evening.

Joel, the democrats have ra... (Below threshold)
DJ Drummond:

Joel, the democrats have ranged nationally from 35% to 39%. They collected 37% in 2004 and 38% in 2006. Republicans have ranged nationally from 33% to 38%. The GOP collected 37% in 2004 and 35% in 2006.

It gets more interesting in the states, where there is greater variance between elections, but even so the state polls which released their weighting have shown heavy tilts to the democrats, especially in Ohio, Colorado, Virginia, and Florida, as much as 17 points greater than the difference in 2004 or 2006. The assumption, for example, that Virginia would go from Republicans +3 in 2006 to Democrats +9 in just 2 years would be literally unprecedented in recorded history.

Obama will win California.<... (Below threshold)

Obama will win California.

That said...if the lack of overt enthusiasm for him in THIS State is any indication (and it may or may not be) then he will have trouble elsewhere.

Last add...in this State the early voting is NOT "landslide" for Obama...even in heavily Dem areas. Anecdotal from a very good source.

Michael McDonald at George ... (Below threshold)
DJ Drummond:

Michael McDonald at George Mason University has early voting data.


Good analysis DJ.T... (Below threshold)
Jeff C:

Good analysis DJ.

There is another aspect besides party weighting that has me questioning the polls. The crossover vote (e.g Republicans voting for Obama or Democrats voting for McCain) being reported does not seem reasonable.

In 2000 and 2004 the breakdown was roughly:

91% Reps vote for Rep candidate
85% Dems vote for Dem candidate

The reason Reps have higher party loyalty appears to be that there are fewer liberal Reps than there are conservative Dems. Dems are more likely to crossover than the Reps, remember the Reagan Democrats?

However, this year the percentages seem almost exactly reversed. In polls, Dems typically show a 90+% loyalty and Reps show numbers in the mid-80 percentages. Can this be real? Can anyone argue that conservative Dems are attracted to Obama and his very liberal record? Are the disenchanted Hillary supporters (the PUMAs) really a non-factor? Gov. Palin has the base fired up as evidenced by the routine 10 to 15K attendance at rallies. The only reasonable reason for a Rep drop is that Black Reps are voting Obama. However, that would only account for a drop of perhaps 2%.

To me it looks like strong evidence of the Bradley effect. You are sitting at home and a pollster calls you. You have no idea who this person is, but he apparently knows you (he has your phone number after all.) He asks are you voting Obama or McCain? A small percentage of respondants take the question to mean, "are you voting for or against the first African-American with a real chance to win the presidency?" They lie or equivocate because they don't want to look bad to the pollster. It is the only way to explain the reported party loyalty percentages.

[13] Don't know if justrand... (Below threshold)

[13] Don't know if justrand is in NoCal or SoCal, but I might add that I've been surprised by the lack of signage for either side this year in my LA County haunts, which run from one end of the San Gabriel Valley to another. Last weekend, I saw no McCain lawn signs in the suburban neighborhood in which I grew up, but also only one Obama sign. Driving around town, I think I've seen just about as many old bumper stickers from '04 as ones for '08.

Did both sides just give up on CA this year?

I haven't seen much adverti... (Below threshold)
Lummox JR:

I haven't seen much advertising here in upstate New York, but then NY like CA is a very blue state--mostly due to NYC--and it's unlikely anyone would waste money on local ad buys. I haven't been paying close enough attention to signage to tell if one side or the other is ahead in this area even anecdotally. I did notice one of the posts here said the numbers suggest NY could be in play, which makes me wonder if we might see a surprise on election night. (Unlikely.) I'd be way more surprised, though, if CA flipped.

DJ,I would only pi... (Below threshold)
kbiel Author Profile Page:


I would only pick one nit with your analysis.

(and let's not mince words - any poll weights by party affiliation, the ones which simply accept what is called in are just accepting the raw data as demographically accurate, which is just as absurd in terms of party affiliation, as it would be if they assumed that race, gender, age, or educational demographics did not need to be reweighted)
Unlike race, gender, age, and education*, party ID is fungible. Assuming no one lies about any these weighting factors or they lie at the same rate for each category, then race, gender, age, and education* have real world data that is absolute (within its precision of measurement). But someone who has voted in the last four Democrat primaries may decide today that they are disgusted with their party and answer that they are a Republican on the next poll call. (And vice verse of course.) Should we assume they are lying? They may intend to change their registration after the general election and they can obviously vote any way they want in the general election.

* Edcuation can obviously change, but only slowly and, assuming the absence of major head trauma, in only one direction.

Now, there are trends and historical data on party affiliation, but, as stock brokers like to say, past performance does not indicate future returns. I see the need to weight by party ID, but unlike the other weighting factors, it can change, quickly and without much rationale (though usually in small amounts). How then, does a polling organization determine the best model? They can not be tied to the numbers from the last election or even the primaries. They can try to read the tea leaves and predict how many will turn out for each party, but they will invariably get it wrong. Perhaps, party ID should just be included in the margin of error in some way. Would that be doable?

kbiel, there is substantial... (Below threshold)
DJ Drummond:

kbiel, there is substantial historical evidence to show that political affiliation changes slowly. Witness the elections of 1976, 1980, and 1992, for example. Turnout decided those elections, but actual party affiliation was generally the same as the preceding and succeeding elections. Nixon's resignation made republicans stay home, but it did not make many republicans become democrats. Similarly, I have more than once pointed out that while Reagan and Clinton were very popular presidents, again they did not change party identification very much at all. There are a lot of reasons for this stability, but the point is that party affiliation should be weighted as a static demographic, because the history tells us that it is such. I have noted that the polls were off by more than their margin of error several times in past elections (1948, 1968, 1976, 1980, 1992, 2000 just for presidential contests), and in every case the polling groups assumed incorrectly that political affiliation would jump in a way it did not.

Obama could win in a landslide, but if he does it will be because he either won over the indies or the republicans gave up and stayed home like 2006. But if republicans turn out and the indies continue as they have been polling this season, the pollsters will be - once again - very embarrassed come November 5, and all because they never tested an assumption that has never been valid before.

Hasn't it always been wrong... (Below threshold)
Fred hill:

Hasn't it always been wrong that being said Obama is almost certainly gong to win

Please help my friend win a very nice guitar by listening to this playlist
(if this upsets you do not get mad at him)
Davids personal favorite C&C

My election range is ---NE... (Below threshold)

My election range is ---NE 17 - IND 21 from subset (A)
...........................NE 24 --- IND 21 from subset (B)
......................and NE 27 --- IND 24 from subset (C)

Will inform as to which subset is the "official" method used, AFTER the game.

From: GALLUP sports handicapping Inc.

DJWell, I agreed t... (Below threshold)
kbiel Author Profile Page:


Well, I agreed that party ID should be factored in some way, but how we factor it is something else. Unlike race, gender, age, and education, which can be checked against other demographic data, all we have with party ID is history.

As I paraphrased before, past performance is not indicative of future returns. Let me give you an example. During my childhood, Texas was very much a Democrat state. In the 80s that started to change and in the 90s, the flood gates opened and the whole state (with the exception of Moscow on the Colorado...er...Austin and the inner cities) flipped dramatically. That this change was balanced nationally with other areas flipping bolsters my point. So, nationally we may see some stability, but there have been dramatic shifts and individuals do change. (My grandparents were yellow dog Democrats until 2000.)

Regardless, my question is how do you weight party ID in a logical manner? Even if you could guarantee that it won't shift more than three points in either direction between any two general elections, there is still a shift that can not be predicted and will not be known until election day. That is why I asked if it was feasible to include party ID in the margin of error, which gives you the ability to have a range, instead as an absolute weighting factor.

But Gallup also says:... (Below threshold)

But Gallup also says:

"Obama has been ahead in Gallup Poll Daily tracking conducted while these data were being gathered. Thus, while equal percentages of Obama and McCain voters have voted early, there are more of the former than of the latter, meaning that early voting generally reflects the same Obama lead evident in the overall sample."

Isn't Gallup saying here that there are far more Obama voters than McCain so the individual percentages are meaningless to the probable result?

Would appreciate your comment on that, DJ.

It's not that hard, kbiel. ... (Below threshold)
DJ Drummond:

It's not that hard, kbiel. Just use the historical record from the last 2 or 3 elections. Actually, you can go all the way back through the 1970 federal elections without getting far from the 38-36-26 DRI split; it's very stable, shifting by less than 3 points in each election. In the case of federal elections, the national split is both reliable and on record.

To bb (23):I canno... (Below threshold)

To bb (23):

I cannot speak for DJ, but while Gallup isn't necessarily lying, I think the quotation you've provided is misleading.

Obama is ahead is Gallup's tracking, in part, because Gallup has given more weight to the Democratic ticket than what is justified by historical patterns. Hence, when they say "there are more of the former than of the latter..." this so-called "lead" is weighted as well. It's a typical question-begging assertion.


bb, in that statement the w... (Below threshold)
DJ Drummond:

bb, in that statement the writer is making a subjective judgment, not an objective note. This is because the actual results of early voting are not released prior to election day, only how many people who vote and in some states, certain demographics about who is voting. The writer is assuming that since Obama supporters say they have or intend to vote, this means that because Gallup is counting more of them that this will result in the same ratio in actual votes, which is not necessarily true. Gallup is assuming its decision to oversample democrats is valid, in direct contradiction to its own observation that republicans and democrats are equally motivated to vote at this time; they are simply using circular logic. Let's turn it around to show what I mean: If I interviewed a thousand people and asked 150 more republicans than democrats whom they supported, even if they had equal interest in the election we would see a strong lead for McCain, wouldn't we? So saying that when you ask more democrats than republicans about their support, that must mean that more democrats will vote than republicans is a false premise on its face. It could happen that way, but its ridiculous to say that bias is ok when the results look good.

DJ,You missed my p... (Below threshold)
kbiel Author Profile Page:


You missed my point. I already agreed that it seems to have been stable on a national level with only a 3 point swing. The problem is that unlike gender, which can be pinpointed by census data, party ID is a range at best. And that is assuming that it does remain within historical norms. No one can predict if it will shift in a two year period, how much it will shift if it does, and in what direction. That is why I asked if it is better or even feasible to include the party ID in the margin of error so that we can capture the whole range instead of guessing what it will be in the current election cycle.

DJ - thanks. I get it now!<... (Below threshold)

DJ - thanks. I get it now!

You have presented fairly c... (Below threshold)

You have presented fairly convincing evidence that there is systematic error in some (or all) or the polls for the general election. The polls were actually substantially in error with respect to the Dem. primaries in many states. This could not have been because of incorrectly weighting party affiliation. Have you looked at the Dem. primary data retrospectively to see what the error in methodology was? Perhaps the same thing is also happening now.

You make a great point abou... (Below threshold)
Lummox JR:

You make a great point about the primaries. I'd be curious about how those numbers break down as well. But on a broader level than just statistical, I think the same thing is happening now: That is, the professional talking heads chose their prom king waaaay in advance and they've pushed their Cinderella-story narrative all year long.






Follow Wizbang

Follow Wizbang on FacebookFollow Wizbang on TwitterSubscribe to Wizbang feedWizbang Mobile


Send e-mail tips to us:

[email protected]

Fresh Links


Section Editor: Maggie Whitton

Editors: Jay Tea, Lorie Byrd, Kim Priestap, DJ Drummond, Michael Laprarie, Baron Von Ottomatic, Shawn Mallow, Rick, Dan Karipides, Michael Avitablile, Charlie Quidnunc, Steve Schippert

Emeritus: Paul, Mary Katherine Ham, Jim Addison, Alexander K. McClure, Cassy Fiano, Bill Jempty, John Stansbury, Rob Port

In Memorium: HughS

All original content copyright © 2003-2010 by Wizbang®, LLC. All rights reserved. Wizbang® is a registered service mark.

Powered by Movable Type Pro 4.361

Hosting by ServInt

Ratings on this site are powered by the Ajax Ratings Pro plugin for Movable Type.

Search on this site is powered by the FastSearch plugin for Movable Type.

Blogrolls on this site are powered by the MT-Blogroll.

Temporary site design is based on Cutline and Cutline for MT. Graphics by Apothegm Designs.

Author Login

Terms Of Service

DCMA Compliance Notice

Privacy Policy