Three Customer Experience Surveying Principles

The Heart of the Matter

We need to create a standard for customer satisfaction surveys. In this post, I propose the following Customer Experience Surveying Principles:

      1. Make it short;

      2. If you ask it, use it;

      3. Never ask a question when a query will do.

——————————

A few weeks ago I met with another consultant offering customer satisfaction surveys, although as only a small part of his business. The conversation turned to methodology when he said “I just like to put together a few questions, and get something out there quickly.”

When I showed shock at his cavalier approach, he argued, “What you have to realize is that these companies are not in the business of doing customer satisfaction surveys. They just don’t want to spend much time thinking about it.”

I was offended at the remark, but held my tongue. What I wanted to say was “They’re not in the business of doing accounting, either. Do you suggest they do a similar half-a** job of that, too?” I simply could not believe he argued for such a deliberately casual and careless approach towards a customer-facing effort.

Unfortunately, he is not alone in this disregard towards interrupting customers. Why else do we find so many terrible surveys? He is casually regarding two pillars that I hold dear: My customers and my brand. How you treat the first directly impacts how they see the second. But apparently this viewpoint is unique.

How else do you explain JC Penney’s satisfaction survey question: “Please select the number 2 for this question.” I get it – they want to validate the scientific accuracy of the response. But what does this say about their opinion of their customers? “We don’t think you’re paying attention, so we’re going to ask a question that shows our low opinion of you.”

Caribou Coffee asks for your receipt number, then proceeds to ask a series of questions to determine what you bought from them. They have your transaction – can’t they query that information? Why make me go through four sequential questions just to tell you that I bought a tea?

I’ve blogged before about a major retailer’s satisfaction survey with its 45 questions, including number 21.1.2.1. What better way is there to say “You have nothing better to do than to answer every question that we can possibly think of?” They have a separate DotCom pickup survey that is 43 questions, preferring 35.1.1.2. The entire survey is 15 pages of text!

Not to be outdone, Tivo (who uses the same survey vendor) has 49 questions in their DotCom satisfaction survey, preferring 24.2.2 as their ridiculous question number.

We need to set a standard for customer experience surveys, whether satisfaction, loyalty, or engagement. These standards are separate from scale, question formation, and other methodological standards. Rather, these standards are designed to minimize impact effort and maximize input when asking customers to invest their time to help you. Specifically, I propose the following:

                Customer Experience Surveying Principles:

          1. Make it short;
          2. If you ask it, use it;
          3. Never ask a question when a query will do.

Make it short. Show that you value your customer’s time by only asking the most critical questions. Realize that satisfaction surveys are distinct from market research – different expectations apply. The primary goal of customer experience surveys is to understand and improve the customer experience – not to discover whether they have two kids at home. Unless needed for segmentation, skip most demographic questions.

Worse, each question reduces the validity of the entire survey. Eventually, respondents just choose the same number over and over again. Rather than trying to get around this by telling respondents to answer “2” in the middle of your survey, it is better to just use a shorter questionnaire.

The rise of the mobile survey reinforces the need for shorter questionnaires. Whether SMS or web-based, very short surveys (5-10 questions) are required. Get ahead of the curve and shorten your survey now. Use your market research efforts, rather than your POS surveys, to ask more detailed questions.

If you ask it, use it. Be diligent – only ask questions which will drive action. In their website survey, TiVo asks “16: *Which of the following best describes your path to tivo.com today?” Exactly how will they use this information? And wouldn’t it be more effective to use Google Analytics for this?

Another retailers asks the following questions in their DotCom order. Please tell me how they will use these questions – and why they feel compelled to ask them in a DotCom satisfaction survey:

*For the next several statements, please select the answer choice that best describes how much you personally agree or disagree with that statement.
24: *When I know exactly what I want, I prefer to buy my technology and entertainment products from a catalog or online
25: *I love to shop for technology and entertainment products
26: *I’d rather go to the store to physically see a product than order it over the Internet
27: *I would be comfortable setting up a home entertainment system by myself
28: *I spend most of my free time doing things with my family
29: *I like to research technology and entertainment products before I buy
30: *I cannot live without the Internet
31: *I only go to technology and entertainment stores when I really need to buy something
32: *I can find the best prices by shopping and buying online
33: *I mostly just follow the advice of the salesperson when deciding what electronics to buy
34: *I’m always thinking about remodeling and upgrading my home
35: *I stay informed about the latest technology and entertainment products even when I’m not planning a purchase
36: *I don’t need to have the latest technology and entertainment products
37: *I prefer to buy my technology and entertainment products at a store because it’s easier to return items there than to a website
38: *I love using technology and entertainment products but I hate shopping for them

What do these questions have to do with their DotCom experience?

Never ask a question when a query will do. This is the toughest for most companies. Why ask a customer to input a convoluted receipt code, then immediately ask for information you should already know? Marshall’s immediately follows the receipt number question by asking whether I made a purchase or a return. In their DotCom survey, TiVo asks whether I made a purchase, and what parts of the website I visited. Walgreen’s asks at what POS I made my purchase. All of this is available through data. Not only can I save my customers’ time by avoiding these questions, I also avoid the risk of using flawed self-reported data, and its lack of reliability.

————–

There was one component of my fellow consultant’s argument that I do agree with: most companies are not in the business of creating customer satisfaction surveys. But they are (or at least should be) in the business of creating customer satisfaction. To these companies, my best advice is: Find somebody who is in the business of creating customer satisfaction surveys. Then use them to measure the experience in a valid manner that positively reinforces your brand.

Follow-Up Question:

I have proposed three Customer Experience Surveying Standards: “Make it short,” “If you ask it, use it,” and “Never ask a question when a query will do.” What standards do you propose?

4 replies
  1. Avatar
    Doug Sheridan says:

    I would suggest that rather than simply make it short, it’s probably better to tell respondents how long the survey will take. We find that our surveys take about 20 minutes to complete. However, over this 20 minutes we are able to collect a tremendous amount of information. So much so that we don’t have to do follow up surveys with them for several quarters. In addition, I have seen studies that suggest lengths of surveys don’t necessarily correlate to completion rates (within reason). My suggestion would be to only include necessary questions rather than simply make the survey short.

    Reply
  2. Jim Tincher
    Jim Tincher says:

    Doug,

    I think it largely depends on the type of survey. It sounds like you are referring to conducting relationship studies – is that correct? I should have been clearer in the distinction between relationship and transactional studies.

    The nature of relationship studies allows them to be longer than transactional. But even then, 20 minutes feels long. I agree that a lot of data can be captured – but is the consumer really focused that long into the conversation? Most of the 20 minute studies I have found (without compensation) find the consumer trailing off after about 10 minutes.

    B2B and compensated studies are exceptions. A good indicator is found in the JC Penney example – if you have to resort to “Please answer 2 to this question” to get good data, then the survey is too long!

    Reply
    • Avatar
      Doug Sheridan says:

      Hi Jim. Yes, you are correct that our surveys focus more relationship studies than transactional studies. However, I would point out that JD Power’s surveys, which are transactional I would argue, are 100 questions long. I do agree that the JC Penney confirmation question is not respectful of the respondent or their time, and should be avoided.

      We tend to make our surveys longer primarily so that we won’t have to go back to respondents a second or third time to get the additional information we’d like. Because we do “compensate” respondents via no-cost reports related to the eventual survey results, the approach of “longer but fewer” surveys works for us.

      Reply

Trackbacks & Pingbacks

  1. […] them, Best Buy’s previous survey made it into my customer satisfaction survey hall of shame (https://heartofthecustomer.com/3-principles/).  In that post, I argued for three customer experience survey […]

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *