Skip to main content

Opinion Polling Bias and How to Avoid Them

Voters will determine the next US president coming months, and polling bias will likely soon become a topic of heated discussion. If the Iowa Caucus taught us anything, polling is still unreliable at predicting the actual results of elections. In my previous posts, I went into detail about polling bias. I followed that up with how pollsters can get create bias in the form of response, researcher, coverage, or non-response. In this post, I’m going to give you some tips and tricks to help you make sure that your polls fall within the margin of error.
Collecting accurate information from polls can be a challenging task, but understanding your audience and writing great content will go a long way. If you have ever read our previous posts, you’ll know that writing an effective poll is an art form. It requires getting the correct information from your respondents in a limited period of time. Here some practical tips to create effective polls.
1.) Find the Right Sample Size 
Generally speaking, most national and state polls have a 500 – 1,000-person sample size. This results in a margin of error of 5% – 3.2%. As the sample size increases, the sample size decreases. However, as the sample size decreases, the returns diminish. So for example, the margin of error decreases significantly more moving from a survey of 50 – 100 than from a survey from 500 – 1,000. Unless tenths of a percent matter to you, your survey sample does not need to be in the ten’s of thousands.
2.) Find the Right Medium to Ask Your Questions
Do you have a landline phone? Would you pick up when you get a call from an unfamiliar or unlisted number? Do you live in a rural area? These questions can lead to survey bias. For example, Republicans are more likely than Democrats to use a landline phone. With the accessibility of the Internet, web scraping or data mining should help pollsters get accurate analytics on national or statewide elections. While it can almost be impossible to get a perfect representative sample, the Internet is almost universally used. If you think that candidates are not already using big data, read this article.
3.) Be Direct and Ask Multiple-Choice Questions 
Multiple-choice questions give the respondent a chance to see all the options, making their task easy. Be bold and make sure that you are communicating your purpose. A good multiple choice question should be mutually exclusive, avoiding ambiguity at all costs. A poor question example is “Describe your political affiliation? A – Democrat B – Republican.” This is a poor question because it assumes that the audience can only affiliate with one of two parties. This will  affect your accuracy because it doesn’t account for Independents, the Green Party, etc., etc.
4.) Speak Your Respondent’s Language
Use simple, direct and specific language in your questions. Start with basic questions and then make it more specific. An excellent poll reads in a way that the reader can understand. And unless your polls are industry-specific, you should avoid jargon at all costs. This can create confusion, which leads to response bias.
5.) No Double Barrel Questions
We know that you are trying to get the most out of your polling. But do not use it as an excuse to ask an informal fallacy such as a double-barrel question. These occur when a question involves multiple subjects, yet has only one answer. For example, “Should the Government spend less on Military and more on Healthcare?” The problem with the question is it’s impossible to see what the respondent is answering. Is the respondent opinionated about one subject and not the other?  Instead, ask two separate questions to get most accurate answers. The best way to answer the above question: “Should the Government spend less on Military?” Followed up by “Should the Government spend more on Healthcare?”
6.) Remove All Bias
Creating leading questions will not represent an honest opinion of the respondent. In politics, loaded questions will especially suppress rational thought and result in knee-jerk reactions from your respondent. What is the difference in response to the questions “Do you support Obamacare” and “Do you support the Affordable Care Act?” Both represent the same law and the majority of Americans are familiar with the terms. Yet, a CNBC reportrevealed that having the word “Obama” both raises the approval rate and disapproval rate.
7.) Great Polls Have a Flow. Rank Questions Accordingly 
Ordering your questions is important to give a sense of flow to the survey. Always rank your questions from simple concepts to complex ones.
What Do You Think?
Why does bias in polling occur, and how can we eliminate this bias? Oops made that mistake. Here’s a better way to ask the question: Do you believe bias occurs in political polling? If you answered yes, what is the best way to eliminate polling bias? We want to hear from you, share this post in social media and let’s start a conversation. Learn more about how to build and deploy online polls at QuestionPro. 


Popular posts from this blog

Results of the Presidential Job Index Survey (week ending 7/1/2010)

Quick update on the results for the presidential job index survey for the week ending 7/1/2011. As a reminder this is a weekly survey that we conduct and we publish the results for the survey every Friday. To participate in this week's survey, please download the SurveySwipe application and take the survey.

Remember, you get points that you can redeem for various rewards when you take our surveys. We have already distributed several paid apps like Angry Birds for free.


Almost 900 respondents took the survey.
52% vs 53% last week of the respondents approve of the job that President Obama is doing.
Respondents included 28% Democrats (31% last week), 25% Republicans (22% last week), 24% independents (27%) last week and 2% with the Green Party (3% last week).

You can view the reports here or directly on your phone.

Top 5 Infographics of the Week: Cyber Security

Earlier this week, we experienced a power outage with our data center at Internap. During this short period of time, our customers were unable to access the site. Luckily, it didn't last long and we are back up and running now thanks to our dedicated support team that responded quickly and worked diligently.

Cyber security is, and always has been, a very important issue for us as we continue to grow and meet the demands of a technology-driven society. Nothing is more paramount than the safety of your valuable data. And apparently we weren't the only ones affected, as LinkedIn was experiencing difficulties on the same day and Internap had 3 massive outages earlier this year. Check out these infographics with stats on the importance of cyber security and the impact it can have on small-medium sized businesses.

Top 5 Infographics of the Week: Internet of Things

In 2008, the number of things connected to the internet exceeded the number of people on Earth. By 2050, we are expected to surpass 50 million different devices connected to the internet. And we're not just talking smartphones and tablets - we mean everything. Wearable tech, household appliances, smart homes, public transportation, hospitals and even cattle have all been using sensors to transmit data automatically. It is an exciting yet scary time to be alive. The various interactions between these different entities are opening up doors for more apps, more software and more auxiliary services. With so many "things" connected online, the possibilities for information exchange becoming wider everyday. Check out the top 5 infographics to learn more about the IoT.