The Lingering Question of the Survey Revolution

The world of market research is in something of a revolution. Firms new and old are revisiting long held beliefs about the effectiveness of their most fundamental tools. Since our industry’s infancy, the survey and the focus group have been our bread and butter, but more and more these tools are becoming outdated, and for several overlapping reasons. On one hand firms are seeing that surveys are often too broad or too biased to be completely relevant, which results in providing clients with unactionable and therefore unvalued data. At the same time the ease and accessibility of free online surveys allows those clients to circumvent firms altogether and collect/interpret that simple data streams themselves. And if a client can lower their overhead while maintaining results, who can blame them? Nowadays the most successful firms are those that can provide something else, tools to analyze and distill multiple data streams to return the results our clients expect. Firms who have embraced text analytics, MROC and chat responses, crowdsourcing, anthropological approaches, or gamification models are some of the most innovative and exciting around. However, amid all this excitement are some lingering questions, fundamental difficulties these new techniques have to consider, problems as old as our industry itself.

In a recent article, D. Daniel Ziv looked at ways in which the traditional customer survey is problematic. Many of the issues he identifies are the ones these new collection strategies are seeking to (or have) solved. For example, what he calls the ‘one size fits all’ survey questions are exactly what online forum collection is addressing. Open-ended collection allows less organizational bias (a sentence about why a customer likes a business instead of a simple yes or no). Further, the problems of gathering responses too far after the fact, or worse, not acting on those responses, are directly addressed by instantaneous, highly specific interactive techniques.

Ziv does, however, mention a few other problems. His critique of incentivizing surveys – the ‘skewing the type of people that respond’ by attracting those with less money and therefore of less interest to a client – while not a criticism I personally agree with, does raise the interesting question of how to get customers to respond in the first place, without which the most sophisticated, dynamic analytic program in the world isn’t worth squat.

It’s interesting to note that some firms (and their big clients) are sticking to the traditional data collection methods. In another article, Kevin Lonnie notes that often interest in social media research comes from management and clients and not research groups themselves. While Lonnie does think the time of surveys and groups has passed, and that text analytics or neuromonitoring are their replacements, he says that new kinds of surveys are still needed to ‘cross-validate’ the white noise of social media findings, which (contrary to Ziv) need to be engage customers by gamification. So again, under a slightly different guise, we see the problem of getting customers into the process in order to make new collection methods as powerful as possible rearing its understudied head.

Consider a third article by industry giant Leonard Murphy. He went about interviewing and investigating three of the most innovative firms in the field – Join the Dots, Motista, and Decooda. For Join the Dots, Murphy has nothing but praise for their integration of multiple data streams – online communities, mobile surveys, website usability – as a way of staying relevant in this changing MR paradigm. Indeed, Join the Dots was recently rebranded for connecting these streams, preferring to call themselves an ‘insight consultancy’ instead of a ‘research supplier’. Similarly, Motista has embraced a powerful combination of cultural anthropology and mathematical modeling to determine what motivates customers, allowing clients to build effective campaigns and achieve competitive differentiation in an era where clients (understandably) demand these results. Or the text analytics pioneer Decooda, who use advanced semantic tools to structure terabytes of mined data from tens of thousands of verbatim surveys within minutes. Three revolutionary companies building strategies from the ground up, embracing quick-paced technologies from multiple external fields to provide never-before-seen value to a whole new playing field of potential clientele. But in these interviews, including these companies’ press releases, answers to the question posed by Ziv and Lonnie are nowhere to be seen.

This is not to say that new technologies aren’t attracting more responses than ever. Indeed, social media collection will get more feedback than a $100 lottery ever could. The point here is that while addressing a huge, huge number of problems in the MR field – bias, relevancy, stream integration, customer community – these new technologies are not specifically focused on the problem of initial engagement. They are immense tools for understanding and presenting what customers are saying, no question, but in the future, as the MR revolution continues, firms need to focus that same level energy and brilliant innovation on ways to mobilize and incentivize those customer responses in the first place. A lot of work has been done so far, but as is always the case in business, there’s still a whole lot more to do.