People throw out a lot of strange ideas at times. Some of that is because people can believe some very silly things, and some of that I blame on television and comic books. But it really gets strange when we see fantasy and illusion play out in politics, in the guise of fact. CBS tried to smear President Bush with documents they admitted were fakes, the claimed inexperience of a Vice-Presidential candidate is loudly harangued, while the weaker resume of the other major party’s Presidential nominee is ignored, and biased opinion polls are touted as objective sources of news. Truly Alice-in-Wonderland stuff, except that this is the real world, and we dare not risk trusting the Mad Hatter.
Opinion polling is fun in many ways, useful in certain aspects, but in the end should not be trusted as a guide or counselor for course decisions any more than one might trust a Ouija board. The bias is often missed, even though it is obvious once you know where to look. I have shown before that every poll is biased to some degree, and I have repeatedly made clear what you should expect from a valid poll:
• public access to internal data
• a consistent, publicly reported methodology
• weighting according to Census norms or reasonable objective standard
• archive data available for comparison to current polls
Alas, there are areas where even the best of opinion polls cannot meet the standard of true objectivity. The most common stumbling block comes when a poll weights its political party affiliation. Polling groups understand that a truly random system of contacting poll respondents will produce results which are, to some degree, at odds with the true opinion of the public as a whole. To correct for this, respondents are asked certain questions to determine key demographic data, and their responses are categorized according to those demographic keys and the overall results weighted so that the response pool model is in line with demographic norms. The demographic standard used is almost always based on the most recent US Census data, for reasons that this data is considered the most unbiased and reliable demographic data available. So, Census data is used for gender, race, age, education, employment, and economic strata norm determination. This makes a lot of sense, and I applaud the pollsters for that standard and diligence in its application. The same polls who are so careful to avoid bias in most demographic norms, however, get completely squirrelly when it comes to party identification.
Political party identification is a stronger factor in candidate support than any other cited demographic category. It is no shock, after all, that democrats overwhelmingly support the democrats’ nominee, while republicans overwhelmingly support the republicans’ nominee. The problem comes in, when the poll weights the response pool to match a desired party identification standard. I noticed four years ago when I looked at the opinion polls, that the polling groups changed their party identification weights, some every week! When I contacted the polls about this practice, most just ignored me, but a few did respond. I found their explanations troublesome, however. Pew, Gallup, and Rasmussen, for example, choose their party weighting by examining the self-reported party affiliation of respondents from a prior period, usually a month (the ones who change each week generally use a rolling average of respondents from the period reviewed). Unfortunately, the reasoning behind such weighting is directly contrary to the whole purpose of weighting in the first place. To see what I mean, let’s apply that logic to other demographic groups.
– continued –
]]>< 