10 Reasons Why Most Market Research Fails


Introduction: 

The promise of market research is that it will deliver actionable customer insight, solid guideposts to positive action companies can take to improve sales, profits, services, products and price performance. All too often, however, the projects end up sitting on the shelf, expensive doorstops. Atenga has identified ten sources for this all-toocommon failure. These are flaws in project design, execution and interpretation. This paper is intended to give managers a reverse checklist, a quick look at their project to make sure none of the errors have inadvertently crept into the process. Better research leads to better execution ... and better results.

These are the errors we have seen:

1. Simplification

2. Isolation

3. Evaporation

4. Homogenization

5. Competition

6. Amateur

7. Confidence

8. Commoditization

9. Transparency

10. Internal vs External


1. Simplification Failure:

When your customers decided to buy from you, or not, they did so based their perceptions of your company and your competitors, convenience, brand image, their willingness to pay, inputs from influencers, offers and promotions, and a variety of other attributes. As we all make these decisions all the time, we don’t deconstruct these various aspects of the decision making, but it is there all the time.

Despite the complexity of these buying decisions, there have been numerous attempts to simplify the capture of customer insights. The most successful – and arguably the most egregious-- is the customer satisfaction measurement “NPS,” the Net Promoter Score. This singlefactor measure of customer satisfaction relies on the undeniable truth that having customers rave to their friends and colleagues about your products and services is a terrific thing. Consequently, if you have a high NPS score, it is a good thing. But if you have a low score, it does not tell you what to do, how to remedy the situation. NPS obscures many of the very real benefits of knowing what will attract the most productive customers, and to what extent you and your competitors are delivering it.

Similarly, I think most of us have experience with short simple surveys; either on-line, by mail, handed to you as you do your purchase, or in a quick phone call. These short questionnaires vastly oversimplify the decision processes, evaluations and judgments customers make regarding your products and services and those of your competitors. The efforts companies spend on these very simple projects are largely wasted and provide little, if any guidance on how you can serve your customers better.

 

2. Isolation Failure:

The Isolation mistake occurs when the customer intelligence process ignores the very real influence of pricing and other considerations. Customers’ perceptions of value must always be assessed in the context of how much they are willing to pay, what they are willing to pay for, how well your price structures and price levels align with their value drivers, and what competitors may be promising for a lower (or higher!) price. Isolating satisfaction from price presents an unbalanced and incomplete picture of buying decision behavior, customer satisfaction, loyalty and willingness to continue buying from you.

 

3. Evaporation Failure

The evaporation mistake occurs when customer intelligence data is collected, analyzed and disseminated, read, and then ignored. Evaporation happens when the information gathered is not actionable. It happens when there is no formal process for evaluating it and acting on it. When the data evaporates, findings may provide informational insights, but they fail to drive meaningful improvements, changes or developments. Of all the errors, evaporation is the saddest; important opportunities for executives to drive meaningful improvements in their business results are squandered.

 

4. Homogenization Failure:

Another undeniable truth is that the market-scape must be measured in the context of market segmentation. Segmentation impacts the process in two very profound ways. 

First, the different market segments you serve will be different in their appreciation of your products and services. Finding an effective segmentation model and a list of significant variables is an art in itself. In some cases the most important variables may be personal – age, gender, weight, socio-economic status, demographic markers. In others, it may be business-related-- industry, title, age or size of company, etc. In still others, the relevant variables may be time-based. Think of variations between morning and afternoon customers, or summer versus fall, or week-day versus week-end. There is enormous value that is lost when the data is homogenized and obscures these important differences.

The second impact may be even more profound. That is that different market segments may make their buying decisions, and be satisfied or dissatisfied, for entirely different reasons. Yet it is those differences, and those reasons, that make the data actionable. Just knowing whether they are satisfied is not enough.

 

5. Competition Failure:

The competition error is one of the most serious. It occurs when the customer intelligence process is limited to just the relationship between a company and its own customers. This limitation ignores the alternatives that customers have, and it gives a false reading on the company’s level of risk. The competition error can be avoided by surveying a wider population than just your customers -- surveying and interviewing your prospects, lost-business contacts and individuals who may have never had any business dealings with you at all, but fit the profile of those who should be interested in your offerings. Their perspectives on your company and your value messages and value drivers, and those of your competitors, create a context within which your own results should be interpreted.

It is only when you know the level of satisfaction, and dissatisfaction, of your competitors, in detail, that you can leverage the effort to drive increased market share.

 

6. Amateur Failure:

The widespread availability of inexpensive survey software has seduced even the most skeptical executives into thinking they can capture their own customer intelligence data - and be effective. The fact is that effective survey design is difficult to accomplish if you are going to create consistent, actionable information. Even the most intelligent, business-savvy, seasoned managers and executives make the mistakes listed in this document. Development of a useful survey takes more than intelligence and software that is easy to use. It takes experience and expertise. Designing a useful customer intelligence instrument is a science as well as an art.

Many of these (often) free survey software services also do not have the more advanced and complex functions needed to drill deeper into the mind of the marketplace. They are unable to analyze how customers and noncustomers trade off features, functions, services, satisfaction and price as they compare different vendors and alternatives. Even if the software had the requisite functions, the likelihood that an amateur would be able to effectively configure the question sets and correctly interpret the results are slim. 

 

7. Confidence Failure:

The Confidence error occurs when the number of customers, and non-customers who disclose their perceptions, drivers and preferences, fails to meet the threshold of statistical significance. When the conclusions rest on a base of too few respondents, they are certainly suspect and often wrong. You need statistically valid data from customers and non-customers, former customers and people who might not even know your company exists, but fit the profile of your buyers. Only then will a customer satisfaction and retention study provide its true value. And to achieve that, there is no substitute for the hard work of getting the right number of survey-takers.

 

8. Commoditization Failure:

In a commoditized marketplace, buyers perceive the product and/or service choices to be so similar that the purchase decision is made mostly by price. The Commoditization mistake occurs in the survey design process. Specifically, it occurs when the survey does not isolate and test the specific differentiators that the company believes distinguish it in the marketplace. Commoditization taints the customer satisfaction survey process when the survey fails to assess the extent to which the client’s (self-perceived) unique value drivers actually underlie customer choices, satisfaction and loyalty, and drive repeat (or ongoing) business.

 

9. Transparency Failure:

The Transparency mistake occurs when respondents are allowed to know the identity of the sponsor of the customer intelligence process. Extensive experience and careful experimentation have shown that customers answer questions differently when they know your identity. This is a major source of error and bias in most customer satisfaction surveys. When respondents know the survey sponsor’s identity, they:

- Bias their satisfaction levels upwards (out of “courtesy”) to the sponsor.

- Bias their answers towards what they think the company most wants to hear

- Bias any indication of amounts they are willing to pay downwards in a desire to drive down the price.

- Misrepresent the status of individual elements of your offering in terms of whether they are “premium,” “nice-to-have,” or “absolutely essential.”

 

10. Internal vs External Failure

All companies have perceptions of what drives their marketplace. The key factors are those that drive customers to select your company in the first place, why some select a competitor, and what drives your customer to become satisfied and to return. Mostly these factors are un-documented and not disseminated throughout the company. They becomes a corporate “gut feel” that varies among different groups in the company. Executives, marketing, sales, product development, customer support, etc. each may have a different “gut feel”. And they may all be inaccurate…. Not completely wrong, of course, but not completely right either, and, specifically, not detailed enough, not ranked and not quantified.

The gaps in perception between the company and the marketplace, and those between different groups within the company, all render an organization less effective than it could be. A company with significant perception gaps will likely miss the mark in terms of persuading new companies to become customers, and on delivering the superior customer satisfaction experience necessary to rise from the sea of me-too’s to become a true market leader.


Summary:

The decision landscape is the terrain a buyer must traverse in order to decide whether or not to buy your product or service. If you are selling to companies, the decision landscape is the approval and influence structures around the buying decision. If you are selling to individuals, it is the set of considerations that determine who they will purchase from, how much they are willing to pay, what they will buy, and the level of commitment to your firm. Customer insight is a wonderful lens through which to map the decision landscape, and plan the actions to drive the business forward. Yet many companies develop products or define their services based on internal considerations, not what drives customer satisfaction or buying decisions. They then pay the price of long sales cycles, reduced repeat business, low profitability and limited growth.

If you recognize any of these customer intelligence failures in your organization, please give us a ring so we can discuss how to help you out.