Have You Ever Caught Yourself Being Biased in Analyzing Your Survey?

Have you ever noticed that when you're in a good mood, you listen differently. You react to circumstances differently. It's as if nothing can get you off balance. If someone cuts you off on the highway, you take it in stride. If your friend or spouse give you a sideways glance, you don't even think about it. And then, there are the "bad" days. Where it seems nothing goes right and those very same things that you didn't even notice before now seem like a personal assault on your very being.

Welcome to the normal functioning of your brain. For every bit of non-emotional fact or piece of data that your brain receives, it attaches an emotional component -- all the better to retain it and make sense of it. That's really what the brain does. It observes a pattern, attaches an emotion and then assigns it meaning.

Usually this is very useful. It's how we're able to get through the day in 24 hours instead of 24 years. And at the same time -- it can create some interesting situations when we're processing survey data.

Bias Happens At All Levels

We often talk about sampling bias or response bias, but have you ever sat down and talked about analytical bias?

The next time you're sitting down with your team reviewing the latest customer satisfaction scores - stop and notice where bias comes in. It sounds something like this:

"It's no wonder our scores are low last march. That was when the new software went live -- it was a mess remember?"

"We just sent out our latest price list. That region got hit with an increase and they also got the survey invitation the next day -- no wonder the numbers are low."

The implication is that we know what happened, we can explain it and that makes it ok. The truth is -- we don't really know what happened. All we know is that we sent survey invitations out on certain days. Respondents gave us lower scores during those times. There could have been any number or reasons.

The point is that our natural bias can cause us to miss something that might be just below the surface -- just AFTER that point where we've put forth the "reason."

Look. I have no facts to support this -- I want to hear from YOU. Have you ever caught yourself unconsciously introducing bias and what were the results?