Your Quest for Survey Data May Be Hurting Your Company

One thing we CX-ers have in common: we love our metrics. Go to any CX conference, and the room that’s filled to overflowing is probably talking about metrics.

Metrics are comfortable for us. Whether we’re talking Net Promoter Score, Customer Effort Score, or good old customer satisfaction, survey metrics give us something to share with the business. Even better, in a role that is so focused on intangibles, we have one tangible thing we can point to.

Of course, to get these metrics we need surveys. Lots of surveys. Long relationship surveys, short (but frequent) transactional surveys, and medium-length touchpoint surveys. More data to analyze and report. We need to feed the beast.

I recently ran across some research that suggests that the continuing search for these metrics may actually be hurting your company.

What has Research Shown?

In this article, Rethinking Market Research’s Place in the Business, Vikas Mittal, Ph.D. (a member of the faculty at the Jones Graduate School of Business at Rice University in Houston) shares some research results from “A 2017 study of 7,513 customers of a large North American automotive dealership.” He doesn’t cite the company. (Ironically, when reading the article online, the AMA website asked me to rate how likely I am to recommend this article!)

His first point is that those who fill out a survey spend more than those who do not; he reports that, “sales per visit were $12.18 greater among those who filled out the satisfaction survey [versus those who did not].” He then argues that this shows a business benefit to surveys.

I’m not convinced. Mittal makes a causal assumption that I don’t share – that the survey caused the increase in spending. It seems much more plausible that the type of person who responds to surveys is different than the type who doesn’t. The person who responds to surveys is likely more engaged with the brand, and thus spends more (although I do acknowledge that there is academic research backing up the causal link between surveys and increased purchasing behavior).

Regardless, his second point is much harder to dismiss: customers’ purchase amounts decreased the more surveys they responded to. This is a big deal. The continued linear nature does suggest causality, which also reflects what I have seen. The more you annoy a customer with repeated surveys, the less engaged they become with your brand, translating into reduced purchases.

Yes, by over-surveying customers, you may be costing your company revenue.

So, what’s a CX-er to do? I have three steps as a recommendation:

1. Reduce the chance of survey overload. Ensure that no customer gets more than one survey per quarter; preferably no more than one every six months. If you don’t have the means to ensure this, then shut the survey down. We’re customer experience people. Our top priority should be engaging customers. Any disengaging activities need to stop.
2. If necessary, reduce the frequency of your reporting. It may be that reducing the frequency of your surveys means you don’t have the sample size to report as frequently. So be it.
3. Fill this void by moving your reporting away from survey metrics to business KPIs. This is what your business partners are more interested in, anyway. Tie your survey results to these business KPIs, showing the linkage between your metrics and the business results.

Now that you’re likely reporting less often, that will give you time to build this linkage. Show how improving your scores lead to real results, and include metrics such as order size, frequency of order, and customers lost and gained in your reporting.

And if you can’t link your surveys with this business data? Then reporting on your survey scores probably wasn’t that important, after all.