The Benefit Real Life Testing Outweighs the Payoff of Looking Good

Having grown up in the more traditional pen and paper world of marketing, I have to constantly remind myself that it's OK to make a mistake.   I have to keep reminding myself that not only are most marketing tests or published pieces not carved in stone -- they aren't even printed on PAPER!  So what am I freaking out about?

Testing by Launching

I'm not advocating rushing out into the world with just anything, but when you're at the 80% mark and you need to do some testing -- don't be afraid of testing live or pre-launching your product or service.

That's what Survey Analytics did with their new SurveySwipe product.  They had developed a mobile market research platform but needed some real life input to tweak the bugs.

They weren't shy about it either - they decided to pick the State of the Union address as their test.  I think that was BOLD and very visible.  The risk to most of us traditional marketing types might be "BUT WHAT IF WE FAIL"  what if people can't use the tool?  What if they think badly of us?  What if... What if...

Well - what if we TELL them it's a test?  And what if we expect that there will be issues and what if we approach the whole thing as a game and as an exciting learning opportunity where we're are all learning something together?  That would be a whole different kind of event.

The result might even be a more energized and excited consumer base -- one that accepts that this is a test and can't wait to see the bugs worked out.  WHAT A DIFFERENCE!

What Survey Analytics Learned

• Mobile Platforms are Definitely Exciting. Based on the overall engagement -- this is HOT HOT HOT!


• Email or push - not both. This is a valuable lesson for all of us - how does your market want to be notified?  Be sure to ask what they want and then do it exactly that way.


• Don't be afraid to wing it. This is my favorite lesson.  The Survey Analytics team KNEW that one of the features was getting real time data.  Instead of playing it safe - they started PLAYING and polling in real time with their beta testers -- asking questions on the fly!

You can read all the details of what they did well and what the Survey Analytics team learned from their SurveySwipe beta test.  But the real lesson here wasn't in the technology, but in the mindset.

Today's technology makes it virtually impossible to really test anything to perfection.   In fact, the standard practice for web content is to actually create several versions of the content and split test using tools like Google Web Site Optimizer.

Wondering what to charge for a product or service?  Chances are you can post a description and prototype on ebay at a variety of prices and see which sells best.

It's clear that the old days of testing something to death have been overtaken by actually putting it out there and working out the kinks in real time.

How do you feel about that?  What are your tips, Do's and Don'ts about when it's ok to test a beta in real time and when more traditional pre-launch analysis is required?