Thursday, March 31, 2016

What Is The Right Sample Size for My Survey?

How many respondents are really enough?
There are two schools of thought about sample size – one is that as long as a survey is representative, a relatively small sample size is adequate. Perhaps 300-500 respondents can work. The other point of view is that while maintaining a representative sample is essential, the more respondents you have the better.
This is a big issue because it impacts all sorts of decisions including the length of your survey, your collection mechanism, and, of course, you’re sampling rate.
So what’s the right answer?
If you aren’t integrating your survey data with web behavioral data, then a relatively small sample size might be okay (I’d emphasize the might). But if you want to combine behavioral analysis and survey data, then forget a sample 300 or 500 respondents. Those numbers simply won’t work.
Let me give you a real-world example showing why that’s true. We’re working right now with a client that samples approximately 1000 site visitors a month for their satisfaction survey. They asked us to do a study of the impact of using one of two internal search tools on their site on both overall site satisfaction and visit accomplishment.
On this site, search is used in about 10% of visits. Since the site gets more than 10 million visits a month, that still yields a heckuva lot of behavior to study – more than 1 million search visits every month. No problem there.
But our representative sample only captured about 100 respondents who’d used search.
Between the two search tools, one served about 70% of the queries. So for the second search tool, we had about 30 respondents to deal with. Getting the picture?
For our analysis, we wanted to track visit reason vs. satisfaction vs. outcomes for searchers. With some visit reasons only accounting for about 10% of visits, there were cases where we were supposed to analyze the outcomes for all of 3 visitors.
And that’s with a survey size of 1000 and a relatively simple cross-tabulation of visit intent and one fairly common behavior. Sure, we could add lots more months to the picture. But tracking behavior over extended periods of time adds all sorts of complications to the analysis. The combination of seasonality, site change and macro-economic change make this dangerous. Very few of our client sites remain constant for six months.
If doing behavioral analysis with 1000 survey respondents is challenging, imagine what it would be like with a sample size of 300. Impossible.
So what’s the right answer?
If I had my druthers, I’d recommend that high volume sites strive for a much higher sample size – something like 15K would be nice on a monthly basis.
Is that too much for your user experience to bear? Obviously, the answer depends on your site volume and your take-up and completion rates.
You can’t do much about volume, but if your survey length is impacting your take-ups or completion rates, then I’d be willing to sacrifice a whole bunch of questions to get to the increased size. The fact is that on many 30-40 question surveys, we’d only expect to use at most 5-10 of those questions in a behavioral analysis. I’d bet even money that your analysts feel that same way and that a heavy majority of questions on many long surveys hardly ever get studied at all.
At some point you may have to make a decision: do you want a whole lot of really shallow information or do you actually want to do analysis on a narrower set of data?
So when it comes to behavioral analysis combined with survey integration, the right answer is pretty obvious. A representative sample is essential, but size really does matter. Let yourself get talked into a 300 person sample, and you might as well throw all that work you did to integrate online survey data with behavioral data in the junk pile.
Here are some more tips on how to determine your survey sample size.

Tuesday, March 29, 2016

Get Creative With Your Surveys and Blow the Roof off Your Business

Online survey tools have really grown up over the years. It’s time for our understanding of what we can do with online surveys to grow up too.  There are so many creative things that you can do with online survey tools that don’t even look like surveys, but that can really blow the roof off your business. Here are just a few:
1. Online marketing:  Bet you never thought of that!  Surveys are an awesome online marketing tool and the really cool thing about QuestionPro’s integrated survey platform is that you can connect your online survey to your website and embed it in your email marketing messages — all the better to build that close relationship with your audience and get to know them better.
2. Social sharing: You can also extend and build your social profile by using more online surveys and polls. People love taking them and sharing them.  My only tip is to change it up a bit here and there – do both fun polls and surveys as well as serious ones.
3. Infographics: The online marketing gurus are always talking about “repurposing” content.  Well, you can take your online surveys and just click on the Infographic button and BAM!  You’ve got yourself and infographic that is ready to post and share online.
4. Selling more stuff: Your QuestionPro account can link to your SalesForce account and that means that you can take your customer survey results and connect them to actual customer contacts.  That is a huge benefit to your sales folks.  When your sales people can actually see how your customers responded to the surveys you can set yourself apart by having a good understanding of what they need.
5. Lead generation:  One of my favorite ways to use my online surveys is to collect leads at trade shows.  You don’t need an internet connection if you use the mobile SurveyPocket platform.  And if you want to get fancy, you can have your tech folks use the QuestionPro API to have your survey forms connect to your email marketing program.
6. Become an industry expert and resource:  Bet you never thought that you could make money using your online survey platform! If you run surveys within a specific industry, you can create powerful reports and graphics that you can own and sell to other members of your industry.  This will position you as both an industry expert and resource.

Thursday, March 24, 2016

Write Survey Questions that Gather Insights You are Looking For

Collecting information for your research can be a challenging task, made easier by good surveys. Writing effective survey is the art of getting the correct information from your respondents in a limited period of time. Remember, it is important to save their time while getting reliable and clean information for your research. There are some important tips to keep in mind for writing great survey questions.
Watch Your Language
Use simple, direct and specific language in your questions. Start with conversational questions and make them more specific. The point here is that the reader must understand it easily. Also, avoid jargons and double negatives.
Improve Reliability of Survey Answers
Do not ask double-barreled questions; instead, ask two separate questions to get more accurate answers. Avoid dichotomous questions, as many might not agree with the dichotomy. Know the knowledge level of your respondent and if in doubt, clarify the concept by giving more details.
Remove Bias
Leading questions will only help you confirm your bias and not represent the truthful opinion of the respondent. Loaded questions tend to suppress rational thought and result in knee-jerk reactions in your respondent.
Use Multiple-Choice Question
Multiple-choice questions give the respondent a chance to see all the options, making their task easy. Set up the choice list to cover all option without overlap.
For example, if you asked the respondent – What are the number of hours you work in a day? Choices should not be “2-3 4-8 8-12 hours. In this case, people who work for 4 to 8 hours. 8-to12 hours will have two choices. This might impact your data quality.
Ranking Questions
Ordering your questions logically is very important to give a sense of flow to the survey. Always rank your questions from simple concepts to complex ones.
Using the Out option
Some respondents can’t answer certain questions because they don’t have the experience or don’t know the correct option. For these situations use “Does not apply” option. Another way to give an Out is to accept user input in the “Other” option.
Great survey questions can provide with some insightful data. Here are some more survey design tips to increase response rate. 
Learn more about QuestionPro different survey questions and answer types.

Tuesday, March 22, 2016

4 Tips That Will Guarantee Higher Open Rate For Your Online Survey

Obviously, there are many different ways and channels through which to distribute an online survey, but email still remains the most common form. Email platforms are continuously trying to help you deal with the overwhelming bunch of emails that you receive, thus increasing the likelihood that the online survey you are sending, might get lost on its way, land under promotions tab or in the worst case - in the spam bin.

If you’re a good marketer, you’ll be able to grab receivers attention and get them to open the email you’re sending. If you fail to do so - well, then your marketing campaign is pretty much done, before it even started.

I am sure that I am not the one telling you this first - but the subject line or title are the most important parts of the email you are sending. The subject is the first thing one will see - so spend a decent amount of time to come up with one that grabs attention, and if you can, I would suggest doing A/B testing. Before sending it out, ask yourself these questions:

Does the title spark interest and invite your reader to learn more? Consider yourself first. If this email landed in your inbox, would it grab your attention, and make you want to open it? Think about this well. If you got and email with subject “My boss thinks I am cheating on him,”, would you be tempted to open it? I think the chances are pretty high you would. Especially if the subject line correspondent and informed you that it will talk about employees that used their company’s mail and resource to find better job offers. Title that sparks interest and relevant material.

Try turning something negative into a positive. Yes, emails with negative subject lines tend to have higher open rates. This one, for example, Top 5 Reasons Why Dumb Mistakes Are Teaching Your Competitor Winning Strategies.” This trick works well and is easy to do.

Don’t use CAPS LOCKS.  While we know you want to be heard, but SHOUTING IS NOT THE WAY. Nobody wants to open an email in which somebody is already shouting at them.

Avoid a personalized subject line. There has been a perception going around that including a name in a subject line could help with targeting the market. That is wrong. Instead, the email receivers will most likely think that you are a spammer. Spam experts have a way of getting lists that have the respondent's full name, so if a person receives an email from someone they don’t know, yet it addresses them by name - they will immediately mark them as spam. Thus, avoid using the first name by all means, unless it is really someone you know personally.

Another way of encouraging respondents to take your surveys is by adding a controversial spin to your surveys. Work on the title that could get their attention, and inspires interest to open it. This might actually lead them to answer the survey and sharing their opinions on the subject. The comments and answers might not be all positive and first, and they don’t have to be. But you’ll have an engaged your audience further leading to more interest and potentially higher response rate.

Now that you've got down how to get people to open your email, make sure that your survey is worth opening with having written great survey questions.