Informaticon: How accurate are political polls? from The Wall Street Journal, 3/12/04

Sharon Begley's article, "Is Your Radio Too Loud To Hear the Phone? You Messed Up a Poll," points out that factors other than sampling error distort the accuracy of political polls.

Pollsters attempt to insure that the group of people surveyed, or sample, are a "random, representative slice of the population." But common "nonsampling errors" can make the sample less than representative:

- Polls tend to include too many women, whites, and older people, because members of these groups are more likely to answer the phone.

- Samples that include only 50% women are not representative of the U.S. population, which is 51% female.

- Pollsters often use random dialing of land-line phone numbers to collect the sample. So richer people, who often have more than one land line, are more likely to be called.

- People with unlisted numbers who are reached by random dialing are less likely to participate in the poll.

- If a person reached by random dialing doesn't answer, pollsters sometimes just skip the number and go on to the next one. People who aren't home or don't answer may have some special characteristics that influence their voting patterns, but most pollsters don't publish the "no-reponse rate."

- Pollsters don't call cell phones, so people without land line phones are never polled.

- The five per cent of Americans who do not have home phones are never polled.

- Characteristics not measured by the U.S. Census are never used to determine which categories to include in the sample. (Begley cites the example given by Columbia University statistician Andrew Gelman: "Surly" people often hang up on pollsters, but no one knows the surliness rate in the voting population.)