Wednesday, May 25, 2016

Analysis of Variance (ANOVA) Explained

Surveys are effective at collecting data. However, insights develop after the fact and arise from the analysis we subject the data to. One of those techniques currently on my favored list is the tried and true analysis of variance (ANOVA).

If we are collecting metric data with our surveys, perhaps in the form of responses to a Likert scale, the amount spent on a product, customer satisfaction scores, or the number of purchases made then we open the door for analyzing differences in average score between respondent groups. If we are comparing two groups at a time (e.g. men vs. women, new vs. existing customers, employees vs. managers, etc.) then it is appropriate to use a t-test to assess the significance of any differences. However, if there are more than two groups it becomes necessary to look to another technique.

ANOVA, or its non-parametric counterparts, allow you to determine if differences in mean values between three or more groups are by chance or if they are indeed significantly different. ANOVA is particularly useful when analyzing the multi-item scales common in market research. In the table below respondents in a restaurant, survey rated three diners on overall satisfaction. The null hypothesis is there is no difference in satisfaction between the three restaurants. However, the data seems to imply otherwise.

Larry’s Diner     6.28
Curly’s Diner     6.05
Moe’s Diner       5.33
Overall                5.65

ANOVA makes use of the F-test to determine if the variance in response to the satisfaction questions is large enough to be considered statistically significant.
In this example, the F-test for satisfaction is 51.19 which is considered statistically significant indicating there is a real difference between average satisfaction scores. ANOVA indicates whether or not there is a significant difference, it does not provide, however, direction as to which group is higher or lower. Statistical packages, such as SPSS and SAS, allow the survey researcher the option of selecting a posthoc test which compares groups for individual differences. In regard to satisfaction, Larry’s Diner was the clear winner with an average score significantly greater than either Curly’s or Moe’s. The difference between Curly’s and Moe’s was not large enough, given the number of respondents, to be significant.
The proper use of ANOVA in analyzing survey data requires that a few assumptions be met including normal distribution of data; independence of cases, and equality of variance (each group’s variance is equal). If these assumptions cannot be met then there are non-parametric tests available which do not require these assumptions.

Data by itself is just that. However, when we judiciously employ statistical tests we can create insight that can have a positive impact on our marketing efforts.

Friday, May 20, 2016

Use surveys for on-demand decision making

Surveys seem to always show up as some kind of afterthought -- something like this: "We're thinking about launching a new product -- let's do a survey!"  I know, that's a bit of an over-simplification, but that's sort of how it goes.

Surveys can be SO MUCH more than that!  

Your surveys can be super social engagement tools - if you'd only give them the chance.
QuestionPro gives you these options from the send tab -- where you can share and send your survey via social media:
survey share
Then you can also share your results via social media --
tweet survey

Use surveys to establish yourself as an industry expert

The mistake I'm talking about here is taking your survey tool and survey capability for granted.  Your online survey tool is like the "Arm and Hammer" baking soda of marketing tools.  There is so much more you can do with it.
If you engage with your audience and run frequent and short surveys, you can easily establish yourself as an industry expert.  Do frequent - even weekly surveys and publish and share the results.  Before you know it -- people will start counting on you to deliver these insights.
And soon -- you will be recognized as an expert in your field or your niche.
Granted -- it will take some thinking and focusing on subject matter and you may even have to create a sort of research plan or research calendar.  This is certainly a terrific way to use your online survey as an additional tool inside your marketing toolbox.

Surveys are your on-demand decision maker

Here's another crazy powerful way to use your surveys that will not only help you make better, faster decisions, but will stop a lot of in-fighting in meetings.
Have you ever been to a meeting where everyone gets into what I call the "I think" conversation.  "I think our customers want X" and then someone else says "No WAY -- our customers want Y" -- this should be your trigger to do a quick online survey.  You can literally have this pulled together, launched to a customer email list and get some decent results to guide your decision-making inside of an hour.
I'm astounded at how frequently I think that doing a survey will be "too hard" or "take too long" -- when in reality, we could have had some preliminary results within the hour and not had to waste any time in arguing about it.

Wake UP

The point of this entire article was to tell you to wake up!  You already have access to a tool that will help you make better decisions, become an expert in your industry and do it all in less time for less money.

Thursday, May 19, 2016

Best Practices for Calculating Mean Scores

Working with mean scores gives the researcher access to a wealth of multivariate statistics, but mean scores are not without their issues. Let’s look at a few of them.
First, mean scores are heavily influenced by extreme scores, also known as outliers. These high and/or low scores can pull the mean up or down depending upon their location. This is why variables that focus on wealth, income or prices are typically reported using the median, e.g. the median housing price or median monthly income. Variables with numerical categories such as customer satisfaction or brand awareness tend to be less influenced by outliers. Options for dealing with outliers include removing them from analysis or recoding them with the median value. Either option is not without considerations that need to be addressed. These are beyond the scope of this post.
Another concern is the overall distribution of the datapoints. If half of the survey respondents say they like their coffee hot while the other half say they like it cold, then the ‘mean’ is going to be in the middle or lukewarm. In cases such as this, the mean or average would be misleading. Further commentary by the researcher outlining the distribution of the data is called for. This highlights the distribution and makes it easier for the decision maker to derive understanding.
In survey research, be it market or otherwise, if we report the mean score then we should also report the standard deviation, the median, and possibly the range (high – low) as well. These supporting statistics provide the end-user a picture to better understand the nature of the data. A graphic can also be useful.
Within all consumer data lays variation. This variability can show up clearly when looking at mean scores if you have the right filter applied.
Learn how mean scores are calculated and more about working with mean scores by reviewing Data Analysis 101 – Metric.

Thursday, May 12, 2016

Ask Qualifying Questions to Survey Your Target Market

My husband is a die-hard fan of DC comics. This morning, while having coffee, he told me that he wants to create an online community for DC comics fans. The criteria for this community is that the members should not be restricted to any demographic or gender. Nor should any random person become a member. Now the biggest challenge was how to accomplish this without much hassle?
He had a set of questions ready, based on which it can be decided who can qualify for being a member of this community. The challenge, however, is to design a survey that lets only the respondents who match certain criteria, to become a member. How can this be achieved? Taking a sip of coffee, I said, ‘You need to use – Qualifying Survey.’
What is Qualifying Survey?
Qualifying Survey is a survey designed in an unbiased manner to filter out the respondents who do not match criteria for you desired outcome. If the respondent matches the logic you have created behind the survey, they will be able to become a member of the community.
After I explained what qualifying survey is to my husband, he said, ‘This is a life-saver. I have the questions for the qualifying survey. How do I create the survey and link it to the community?’
Below are the steps:
Step 1. Create the survey (add all the questions and logics) and add a question type ‘Community Recruitment’ which is under ‘Add-on Modules’ from Advanced questions.
Qualifying Survey
Step 2. Go to the settings option for this question, and select the community you want to link this survey to:
Qualifying Survey 2
Step 3. Go to your community and select the option ‘Qualifying Survey’ from Edit. Now enable the option, select your survey and click on Save Settings.

Qualifying Survey 3
Step 4. Your qualifying survey is now linked to your community. You can post the portal link on social media sites, embed it in your website or email it to your friends and relatives.
Below is the invitation for a community – DC Comics Fans – an example I have created for my husband’s project. You can click on the link, and click on ‘Join Now’. On successful completion of the survey, you will receive an email invitation from QuestionPro. Only members who can answer the questions correctly will be redirected to the page asking for information to join the community. Remember, you have to be a DC fan get the questions right in the survey. Answer them and become a member of the community. All the best!

Tuesday, May 10, 2016

Improve Research Skills - Infographic

Having a written questionnaire with a survey platform is just not enough anymore for effective marketing research. In fact, it’s really nothing, not even next to nothing.
  • Survey cooperation rates have plummeted from 43 percent in 1997 to 14 percent in 2012.
  • Online survey responses rates are even sadder. Some studies show participation rates averaging 2 percent.
  • Everyone is taking a slice of the survey pie, including Google and Twitter. This has translated into more competition. Thus, a collective respondent fatigue has settled over the population. As an example, one survey firm, Mindshare Technologies, conducts 60 million surveys every year. That’s an eye-watering 175,000 per day!
These statistics are just scratching the research surface. They don’t even mention the bombardment of information on consumers that has, in effect, shortened their attention span and patience for surveys.
Don’t panic. All of this information just means one thing: more care needs to be afforded to writing questionnaires and the delivery of online surveys. The “more care” is presented in the below infographic, based on qSample’s article 7 Hellish Ways You’re Burning Your Online Survey Respondents.
Really, don’t panic. At the end of the infographic, entitled Survey Puzzle, you won’t be the one puzzled at why your survey data is incomplete and faulty.

Thursday, May 5, 2016

Different ways to use customer satisfaction survey results

With so much chit chat about your business on social media sites like Facebook and Yelp, you may be tempted to skip the formal customer satisfaction survey process.  This would be a mistake.  While watching, analyzing and participating in conversations with your customers on social media sites is an excellent practice, it doesn't make up for have a real program for customer satisfaction.  feedback

1. Use customer satisfaction surveys to measure quality

Customer satisfaction surveys are an awesome tool to measure the quality of your business process  If your organization is using some kind of quality system such as ISO, TQM, Lean Manufacturing, Six Sigma, or any other system design to maximize profits while eliminating waste, then you'll want to lean into this section.
Most quality systems are rooted in understanding true customer needs and then making sure that all of your business systems are focused on meeting those needs with as little waste as possible.  No matter which system you are using, a customer satisfaction program is the foundation.
A core principle of any of these systems is NOT to tell you how to measure quality, it's really to define what matters to your customers and then develop a business and quality system that is designed to deliver the desired outcome.
The key question type you'll want to use for this is the Importance/Satisfaction matrix question.
Simply create a list of items that is a mix of the following:
  • What you think is important to customers
  • What the industry or competition is offering that you don't
  • What customers have told you is important
Then send it out as a survey.  I would recommend that you don't have more than 10 attributes and only add one other question to your survey -- this one:
"What is the one thing (COMPANY) can do to improve your experience with us?" -- make it an open ended question and see what happens.
This alone will help you create a wonderful foundation for your business and quality system

2. Measure your marketing effectiveness with customer satisfaction surveys

Management is always complaining about marketing ROI (or the lack of connection between marketing investments and ROI) the best way to do that is by using a customer satisfaction survey.
The Gallup Organization has a tried-and-true methodology which they call Customer Engagement.  The Gallup Customer Engagement survey consists of 11 questions that measure your customers' Loyalty and Emotional Attachment to your brand or organization.

Here are the 11 Questions Outlined by the Gallup Customer Engagement Survey

  1. Overall, how satisfied are you with [Brand]?
  2. How likely are you to continue to choose/repurchase/repeat (if needed) [Brand]?
  3. How likely are you to recommend [Brand] to a friend/associate?
  4. [Brand] is a name I can always trust.
  5. [Brand] always delivers on what they promise.
  6. [Brand] always treats me fairly.
  7. If a problem arises, I can always count on [Brand] to reach a fair and satisfactory resolution.
  8. I feel proud to be a [Brand] [customer/shopper/user/owner].
  9. [Brand] always treats me with respect.
  10. [Brand] is the perfect [company/product/brand/store] for people like me.
  11. I can’t imagine a world without [Brand].

3. Use customer satisfaction surveys to develop new products and services

It's easy to get tunnel vision about customer satisfaction surveys.  One trap is to look at the surveys as a "grade" or evaluation of your performance.  That's definitely one aspect of the survey, but an even more powerful feature of a good customer satisfaction survey is that it opens up the conversation for developing new products and services.
Instead of just reporting on the results, why not drill down into the results and connect with specific customers to get deeper insights into what's missing for them and their experience with you.  You might discover a real competitive advantage hiding inside your feedback.

4. Arm your employees with customer insights

Your employees can really benefit from seeing the results of your customer satisfaction surveys.  Remember -- the results aren't there for you or the management to beat up on the employees, why not engage them in the conversation about the results and get their side of the story.
You're going to get two benefits from this process:
  1. Your employees will be MORE ENGAGED in the business
  2. They will give you deeper insights about how to actually improve on those results.  Often customer dissatisfaction is a systemic problem and one your employees can't do much about.  But they can pinpoint exactly where to go and what to do to fix the problem.
So there you have it -- I've given you four new ways to use your customer satisfaction survey results.  Here is are also templates you can use for customer satisfaction surveys. 

How do YOU or your clients use customer satisfaction results?  Let us know in the comments below.

Tuesday, May 3, 2016

Use Customer Feedback Surveys as Valuable Sales Tools

Have you ever noticed that whenever it's time for budget cuts, marketing programs, and market research are always at the top of the list?  That's because these activities tend to fall into the "nice to have or nice to do" category of expenses rather than the "must do" category.  I mean, when have you ever seen a company cut down on production or sales when times are tough and they need to get orders in and push orders out?
Well, today is the day I'm going to give you a much-needed contextual whack on the side of the head.  Today I'm going to show you how you can use your survey tool as a valuable "trigger event" predictor that will help you bring sales through the door -- and look like a hero.

Trigger events are all around you and they are sales just waiting for you to close them-- you just have to see them

Craig Elias is the man behind the "Trigger Event" -- or perhaps he calls it "Event Trigger", any way you slice it, he has made a profitable living paying attention to those small actions potential buyers take that signal a big purchase.  If you want more details than I'm going to cover here, check out his blog or better yet, pick up a copy of his book, Shift .
Here's how trigger events work.  There are incidents or events that occur that prompt or trigger us to go looking for a product, service -- solution.  For example, pregnancy is a trigger event, as is buying a house or getting married.  You get the idea; an event happens and suddenly you're in the market for a whole lot of things.

This is where your surveys can come in really handy.
  1. Segment your customer list.  This is critical to the art of customer success. Your customers will be frustrated and unsubscribe if you send them unrelated messages and surveys.
  2. Design several surveys designed to tag and measure your customers' progress along an event cycle.  For each segment, you'll want to design a survey (or surveys) that pinpoint your customers "location" on a purchasing event timeline.  For example; have you had a baby in the last 6 months?
  3. Send surveys regularly to catch an event trigger and market to it.
The real business benefit to your organization will be not only having a closer connection to what's happening in your customers' lives, but actually being able to track and predict where they are in your selling cycle.