Voters will determine the next US president coming months, and polling bias will likely soon become a topic of heated discussion. If the Iowa Caucus taught us anything, polling is still unreliable at predicting the actual results of elections. In my previous posts, I went into detail about polling bias. I followed that up with how pollsters can get create bias in the form of response, researcher, coverage, or non-response. In this post, I’m going to give you some tips and tricks to help you make sure that your polls fall within the margin of error.
Collecting accurate information from polls can be a challenging task, but understanding your audience and writing great content will go a long way. If you have ever read our previous posts, you’ll know that writing an effective poll is an art form. It requires getting the correct information from your respondents in a limited period of time. Here some practical tips to create effective polls.
1.) Find the Right Sample Size
Generally speaking, most national and state polls have a 500 – 1,000-person sample size. This results in a margin of error of 5% – 3.2%. As the sample size increases, the sample size decreases. However, as the sample size decreases, the returns diminish. So for example, the margin of error decreases significantly more moving from a survey of 50 – 100 than from a survey from 500 – 1,000. Unless tenths of a percent matter to you, your survey sample does not need to be in the ten’s of thousands.
Generally speaking, most national and state polls have a 500 – 1,000-person sample size. This results in a margin of error of 5% – 3.2%. As the sample size increases, the sample size decreases. However, as the sample size decreases, the returns diminish. So for example, the margin of error decreases significantly more moving from a survey of 50 – 100 than from a survey from 500 – 1,000. Unless tenths of a percent matter to you, your survey sample does not need to be in the ten’s of thousands.
2.) Find the Right Medium to Ask Your Questions
Do you have a landline phone? Would you pick up when you get a call from an unfamiliar or unlisted number? Do you live in a rural area? These questions can lead to survey bias. For example, Republicans are more likely than Democrats to use a landline phone. With the accessibility of the Internet, web scraping or data mining should help pollsters get accurate analytics on national or statewide elections. While it can almost be impossible to get a perfect representative sample, the Internet is almost universally used. If you think that candidates are not already using big data, read this article.
Do you have a landline phone? Would you pick up when you get a call from an unfamiliar or unlisted number? Do you live in a rural area? These questions can lead to survey bias. For example, Republicans are more likely than Democrats to use a landline phone. With the accessibility of the Internet, web scraping or data mining should help pollsters get accurate analytics on national or statewide elections. While it can almost be impossible to get a perfect representative sample, the Internet is almost universally used. If you think that candidates are not already using big data, read this article.
3.) Be Direct and Ask Multiple-Choice Questions
Multiple-choice questions give the respondent a chance to see all the options, making their task easy. Be bold and make sure that you are communicating your purpose. A good multiple choice question should be mutually exclusive, avoiding ambiguity at all costs. A poor question example is “Describe your political affiliation? A – Democrat B – Republican.” This is a poor question because it assumes that the audience can only affiliate with one of two parties. This will affect your accuracy because it doesn’t account for Independents, the Green Party, etc., etc.
Multiple-choice questions give the respondent a chance to see all the options, making their task easy. Be bold and make sure that you are communicating your purpose. A good multiple choice question should be mutually exclusive, avoiding ambiguity at all costs. A poor question example is “Describe your political affiliation? A – Democrat B – Republican.” This is a poor question because it assumes that the audience can only affiliate with one of two parties. This will affect your accuracy because it doesn’t account for Independents, the Green Party, etc., etc.
4.) Speak Your Respondent’s Language
Use simple, direct and specific language in your questions. Start with basic questions and then make it more specific. An excellent poll reads in a way that the reader can understand. And unless your polls are industry-specific, you should avoid jargon at all costs. This can create confusion, which leads to response bias.
Use simple, direct and specific language in your questions. Start with basic questions and then make it more specific. An excellent poll reads in a way that the reader can understand. And unless your polls are industry-specific, you should avoid jargon at all costs. This can create confusion, which leads to response bias.
5.) No Double Barrel Questions
We know that you are trying to get the most out of your polling. But do not use it as an excuse to ask an informal fallacy such as a double-barrel question. These occur when a question involves multiple subjects, yet has only one answer. For example, “Should the Government spend less on Military and more on Healthcare?” The problem with the question is it’s impossible to see what the respondent is answering. Is the respondent opinionated about one subject and not the other? Instead, ask two separate questions to get most accurate answers. The best way to answer the above question: “Should the Government spend less on Military?” Followed up by “Should the Government spend more on Healthcare?”
We know that you are trying to get the most out of your polling. But do not use it as an excuse to ask an informal fallacy such as a double-barrel question. These occur when a question involves multiple subjects, yet has only one answer. For example, “Should the Government spend less on Military and more on Healthcare?” The problem with the question is it’s impossible to see what the respondent is answering. Is the respondent opinionated about one subject and not the other? Instead, ask two separate questions to get most accurate answers. The best way to answer the above question: “Should the Government spend less on Military?” Followed up by “Should the Government spend more on Healthcare?”
6.) Remove All Bias
Creating leading questions will not represent an honest opinion of the respondent. In politics, loaded questions will especially suppress rational thought and result in knee-jerk reactions from your respondent. What is the difference in response to the questions “Do you support Obamacare” and “Do you support the Affordable Care Act?” Both represent the same law and the majority of Americans are familiar with the terms. Yet, a CNBC reportrevealed that having the word “Obama” both raises the approval rate and disapproval rate.
Creating leading questions will not represent an honest opinion of the respondent. In politics, loaded questions will especially suppress rational thought and result in knee-jerk reactions from your respondent. What is the difference in response to the questions “Do you support Obamacare” and “Do you support the Affordable Care Act?” Both represent the same law and the majority of Americans are familiar with the terms. Yet, a CNBC reportrevealed that having the word “Obama” both raises the approval rate and disapproval rate.
7.) Great Polls Have a Flow. Rank Questions Accordingly
Ordering your questions is important to give a sense of flow to the survey. Always rank your questions from simple concepts to complex ones.
Ordering your questions is important to give a sense of flow to the survey. Always rank your questions from simple concepts to complex ones.
What Do You Think?
Why does bias in polling occur, and how can we eliminate this bias? Oops made that mistake. Here’s a better way to ask the question: Do you believe bias occurs in political polling? If you answered yes, what is the best way to eliminate polling bias? We want to hear from you, share this post in social media and let’s start a conversation. Learn more about how to build and deploy online polls at QuestionPro.
Why does bias in polling occur, and how can we eliminate this bias? Oops made that mistake. Here’s a better way to ask the question: Do you believe bias occurs in political polling? If you answered yes, what is the best way to eliminate polling bias? We want to hear from you, share this post in social media and let’s start a conversation. Learn more about how to build and deploy online polls at QuestionPro.
Comments
Post a Comment