I Get Emails: Can We Trust The Polls?

Here’s an email I received this week-end,

Hey John,

I was just wondering about the accuracy of opinion polls and I’m interested in your comments about it. Most opinion polls, in my view, don’t have a large enough poll sample to be accurate. When pollsters poll 1200 people in a country of 300 million Americans, then make statements like, 60% of Americans or 40% of Americans or whatever, how can we even think to say that they are accurate?

Also, the results can be entirely different depending on what part of the country you are getting your sample from. I agree with the President when he says that he doesn’t pay attention to opinion polls. With that kind of small polling sample, wouldn’t you or anyone agree?

Opinion polls do nothing but inaccurately manipulate public opinion. Don’t you agree?

I would love to see a poll based on at least 100 million people from all different parts of the country.

A dime for your thoughts my friend?


Polls are a complex subject. Here are a few quick thoughts…

#1) It’s okay to have a healthy amount of skepticism about polling data, but conservatives tend to be too distrustful of polls. That’s part of the reason, for example, why we had so many conservatives who thought the GOP would be fine going into the 2006 elections when some of us, cough cough, knew from looking at the polls we were going to be in deep trouble.

#2) Usually, the size of the sample isn’t the problem. If the poll is set up correctly, you can get numbers almost as accurate polling 1200 people as you can polling 100 million (the only difference is that the margin of error would drop down to around zero on the big sample).

#3) Whether the poll is of adults, registered voters, or likely voters makes a significant difference. As a general rule, if I see a poll of adults, I move the numbers 5 points to the right to figure out about where likely voters would come down.

#4) There are a lot of ways to skew a poll. There are two ways in particular that pollsters can do that.

#A) The first is by oversampling Democrats and/or Independents. Keep in mind that even if you are trying to get this ratio right, it’s not easy to do because from election to election, different percentages of each group vote.

#B) The second way to pull it off is by slanting the questions. To name one example, on illegal immigration polls, pollsters tend to give people a choice between giving illegal aliens some sort of legal status and deporting them all, which is a false choice, since getting rid of illegals by attrition, not deporting every illegal, is the preferred “tough on illegal immigration” stance.

#5) You should also keep in mind that polls are just snapshots of opinion at particular points in time. They don’t tell the whole story, they do change over time, they measure the intensity level of opinions very poorly, and it’s not the least bit unusual for people to hold completely contradictory opinions.

#6) The most important thing to remember about polling data is this: one poll is meaningless, but multiple polls aren’t. If you have several polls to work with, from different polling agencies, and the numbers seem to be roughly matching up, you can get a pretty good idea of what’s going on. Over time, you can also watch for trend lines.

Share this!

Enjoy reading? Share it with your friends!