« Microsoft says next Windows won't be as annoying | Main | Corporate Tax Effects 101 »

States and Shadow

Earlier this season, I wrote about the statistical effect of what I call "shadow", the combination of a poll's margin of error and the undecideds. In today's article, I apply this again to the state polls and address the errors of aggregation and over-simplification.

The national polls are showing a tightening race in several polls, notably Gallup, Battleground, and the AP-Gfk poll. Of course, other polls are claiming a large lead for Obama, notably Pew and Newsweek. The state polls are also showing some movement, although its not as rapid for a number of reasons, not the least being that state polling is not done as regularly as national polling. Obama supporters have greatly enjoyed the RCP aggregate numbers for Obama, which they would, since the RCP aggregates at both the national and state level indicate Obama is winning easily over McCain. The problem, of course, comes when you start to look closely at the support for that belief.

If the methodology is sound, national and state polls should track in similar fashion. This does not mean that every state poll will reflect national support to the same degree, but if a national poll is done properly, it will include proportionate responses from every region of the country, ideally from every state, and so the national numbers will reflect the sum of the state supports. So, the tightening of the national race has to mean - assuming the polls are valid - that McCain is gaining support in some large states or in enough small to medium states to be reflected in national numbers. But as I said, major polling is done less often at the state level; most state-level polling is done less than once a month by polling agencies. Survey USA, for example, who has done more state polls than any other agency, has not done a state poll in the last two weeks in 35 states, and has not done a state poll in the last 10 days in 41 states. That's important to keep in mind.

I need to address the problem of aggregation in polling now. Aggregates are popular because they are easy to read, and seem to be helpful in telling how much someone is ahead. After all, you don't want to be fooled by paying attention to an outlier, and there is a sense that if most of the polls say the same thing, that's most likely what's really going on. The problem with that, is the assumption that all of the polls in an aggregate are valid, that all can be accepted with equal confidence. But that would be erroneous. First of all, not every polling group is really professional at what they do. Remember the disastrous early exit polling in 2004? In that case, a lot of brand new pollers were hired and hustled out without proper training, orientation, or supervision. Does anyone really think that was the only occasion where that happened? The fact is, a lot of polling errors get made without the public ever hearing about it, for a number of reasons, not the least being that if their results are what is expected, the error is not obvious. Also, even professional polling groups may look for different characteristics, such as polling adults, registered voters, people who have voted in recent elections for their 'likely voter' category, people who simply claim they are 'likely' to vote, and so on. Take a look at some of these state polls, and you will also find that it can be difficult to see how they arrived at their numbers; many simply do not provide access to the raw data or their internal demographics. As a result, a significant portion of the state polls are likely to be flawed in a functional manner, and aggregating such polls tends to magnify such errors, not eliminate them.

- continued -

The next problem is over-simplification. This shows up most often in the way that polls are reported. Whether you like the results from a poll or not, it's very important to understand that polls are sometimes just plain wrong, and even if a poll is valid, it's only valid to the extent that it demonstrates a trend against its earlier report using consistent questions and methods, and polls have never predicted the surprise results, because they are modeled in a way which reflects the public's assumptions far more often than the actual condition. Polls are opinion polls, after all, not predictors of future events. Polls only "predict" the results of an election to the degree that the voters behave in line with the poll's assumptions.

So, with that said, I am addressing the state polls with respect to the statistical phenomenon of shadow. 'Shadow' is the total amount of uncertainty in a poll, the combination of the undecideds plus two times the published margin of error. For example, let's say candidate A is leading candidate B in a poll, 51-44 with a published margin of error of 4%. Game over, it seems. But that 4% MOE means that either candidate could be as much as 4 points stronger or weaker, meaning its candidate A at 47 to 55, and candidate B at 40 to 48. Also, there are 5% undecideds in the poll, so while B looks to be out of it, it's mathematically possible for the actual condition to be A 47, B 53. It could also end up being A 60, B 40, with the same level of probability as the other extreme. And of course, this does not consider the possibility of some voters changing their minds. I do not think that happens as wildly as the polling groups seem to claim, but it is a valid factor. Considering that, we can now examine the state polling condition.

I took the RCP aggregates (I know, I know, but I do not have the time or space to examine each and every state poll for validity, I don't need to have anyone whining about 'cherry picking' polls, and I can make my point even by using the aggregate reports) and applied the percentages claimed to the 2004 voting results as a two-party vote split. If we count all of the states according to who leads according to the RCP aggregates, Barack Obama would take 50.2% of the popular vote to 43.0% for John McCain, and 364 electoral votes to 174. However, even using those aggregates, the numbers change considerably if we consider the effect of shadow. Applying the shadow rule (undecided plus double MOE), it becomes 200-118, Obama still in good shape but with 220 electoral votes still to be decided.

Before ending this article, I also looked at the trends and outliers in the polling I have seen, especially given certain key internals. I will not call it definitive, but in my opinion if the demographic weighting is corrected the popular vote becomes Obama 46.9%, McCain 46.6%, but with McCain taking the electoral vote 278-260. When the shadow effect is applied, the electoral numbers change to 147-71 McCain, with 320 to be decided. The message is clear then, that the race remains to be decided.


TrackBack URL for this entry:

Comments (9)

DJ, anything opinion <a hre... (Below threshold)
David Marcoe:

DJ, anything opinion these figures from CNN for early voter turnout?:

As of Tuesday, just over 396,000 registered Republicans had cast early votes in North Carolina, compared with registered Democrats, who had cast 771,500 ballots -- nearly twice as many...

So far, early Democratic voters in Colorado have outnumbered Republicans by a smaller margin -- about 6,600 votes.

Election officials in Nevada only report party registration for Clark and Washoe counties, where the major cities of Las Vegas and Reno are located. There, early voters have been trending heavily Democratic: 161,463 to 90,017.

The two counties account for about 90 percent of the state's population, and Democratic turnout is currently about 75 percent higher than turnout for Republicans, according to The Early Voting Center.

Addendum: Could we be seein... (Below threshold)
David Marcoe:

Addendum: Could we be seeing the PUMA effect in early voting?

DJ, most helpful.M... (Below threshold)
Captain America:

DJ, most helpful.

Many don't realize that polls are a lag indicator and not a predictor.

Moreover, I have real problems with pollster like Scott Rasmussen who have been on air heavily in recent weeks predicting the outcome of the election based on his lag indicator poll results.

It is always dangerous when Scott go beyond their areas of expertise (polling) to become prognosticators.

DJ, if you're still wonderi... (Below threshold)

DJ, if you're still wondering what the campaigns' internal polling says, Jim Geraghty has a couple of hints on his NRO blog today. He cites a McCain source as saying that McC is polling 6 points ahead of where Bush was in '04 in key parts of OH, while Obama is polling in PA behind where Kerry was. Also, an Obama campaign official agrees that the battleground states are tightening.

Just sayin.'

I would agree with DJ here ... (Below threshold)

I would agree with DJ here - but Lord knows I am no expert. What I think will pull this election in McCain's direction is the "undecided" and I give most of that 10-12% to McCain. I think that Obama is an emotional choice - people getting swept up in the hoopla. I also think a good number of his supporters have voted (I can't wait to get out and vote for him)and that does not leave many Obama supporters left. Back to the "undecided" voter. They are either not saying OR they are taking a good look at Obama and not falling in line with the emotional aspect and will move towards McCain.

I agree about the emotio... (Below threshold)

I agree about the emotional aspect.
Instead of people being mesmerized by the
popularity of O like he is a hollywood star if some of the people that are educated and undecided realize the truth about
some of the scary connections O has then maybe that will sway the end result with them voting for McCain/Palin instead

2 -- Despite this Dem advan... (Below threshold)

2 -- Despite this Dem advantage in NV early voting, an exit poll only had it at 50-48 for Obama. That's very encouraging.

I see you all have not lost... (Below threshold)

I see you all have not lost your fight :

GOOD! Because we have a lot to do. You! (the American people) are going to have to take back control of your elected government at every level, and set your government back on the right path of service to you, and the greater good of the World.

Barack Obama and the democrats are your best hope of doing that now. Tell your family, friends, and everyone you know to support them as best they can. Because the Bush McCain vote fraud, vote cheating, vote buying, vote manipulation machine is already hard at work to cheat you again. And we all know what a disaster that has been the past 8 years of Bush McCain.

Barack Obama and the democrats will need all the power you can give them at every level of government (Federal, State, County, and local City elected governments). Obama and the democrats will have an enormous mess to fix for the American people, and the rest of the World. A mess caused by the corrupt Bush McCain administration.

You see, starting back in 2000, and before 911, it was mostly the Republican governors, Republican legislatures, and county elected Republican officials that conspired with the corrupt Bush McCain administration to raise college, and university tuitions by the fastest, and highest rate increases in American history. Some state tuitions went up by as much as a WHOPPING! 30% in one year.

The reason the Bush McCain administration did this was to force struggling working class kids into the military to pay for the sudden jump in tuition. Which was forced on them by the corrupt Bush McCain administration, and their corrupt Republican Governors, and republican controlled state legislatures.

See, Bush McCain had plans to get us into all these immoral, foolish, criminal, and unnecessary wars from the start. So they could use these wars to seize power, and later to get reelected. But, for their evil plan to work they needed more volunteer soldiers struggling to pay for an education whose blood they could spill to help them seize more power. Remember Bush McCain's "MISSION ACCOMPLISHED!" theatrics.

The exploitation, and lost lives of these finest Americans is despicable, disgusting, immoral, corrupt and criminal. And it makes me SICK, and ANGRY!

You will have to vote for Obama, and the democrats in overwhelming numbers to overcome the Bush McCain vote fraud machine. Vote early if you can. Then help your fellow Americans cast their votes now, and on through election day. Vote for Obama, and the democrats like your life, and the lives of your loved ones depends on it. Because it does. You will not survive 4 more years of "Let Them Eat Cake" Bush McCain, and their republican allies.

Just look at the mess we have now.

You can fix this mess with your votes for Obama, and the democrats. And REMEMBER, no matter which of us may stumble or fall, the rest of you must continue to surge forward for Barack Obama, and the democrats, and for your-selves most of all. The children, and the World are counting on us.

It's in your hands now. And I know you will get it done.

God bless all of you.


vote for Obama? An empty s... (Below threshold)

vote for Obama? An empty suit with no experience, who even JOE BIDEN says will likely screw up his first 'challenge'?







Follow Wizbang

Follow Wizbang on FacebookFollow Wizbang on TwitterSubscribe to Wizbang feedWizbang Mobile


Send e-mail tips to us:

[email protected]

Fresh Links


Section Editor: Maggie Whitton

Editors: Jay Tea, Lorie Byrd, Kim Priestap, DJ Drummond, Michael Laprarie, Baron Von Ottomatic, Shawn Mallow, Rick, Dan Karipides, Michael Avitablile, Charlie Quidnunc, Steve Schippert

Emeritus: Paul, Mary Katherine Ham, Jim Addison, Alexander K. McClure, Cassy Fiano, Bill Jempty, John Stansbury, Rob Port

In Memorium: HughS

All original content copyright © 2003-2010 by Wizbang®, LLC. All rights reserved. Wizbang® is a registered service mark.

Powered by Movable Type Pro 4.361

Hosting by ServInt

Ratings on this site are powered by the Ajax Ratings Pro plugin for Movable Type.

Search on this site is powered by the FastSearch plugin for Movable Type.

Blogrolls on this site are powered by the MT-Blogroll.

Temporary site design is based on Cutline and Cutline for MT. Graphics by Apothegm Designs.

Author Login

Terms Of Service

DCMA Compliance Notice

Privacy Policy