I read a lot about architects. I always thought the phrase “God is in the Details” originated from German-born architect Ludwig Mies van der Rohe. I was surprised to learn it is generally attributed to author Gustave Flaubert. In either case, I didn’t create it; it still says much if we listen. But how does this apply to satisfaction and loyalty information?
Satisfaction and loyalty data and statistics
Customer Experience (CX) professionals dealing with satisfaction and loyalty worry about statistical validity, margins of error, and 95% confidence. They fear most when copywriters, editors, and advertising people start throwing around generalities, which sound impressive but don’t do justice to the data.
How many of us remember seeing a TV commercial that opened with “4 out of 5 dentists would recommend Trident® gum for their patients who chew gum”? Did you have the same mental image I had (and still have) of 5 people in dental gowns standing in a row with 4 holding a thumb up and the other a thumb down? Well, that was the basis for an ongoing ad campaign. And I must confess that I had to Google the first part of the quote to identify the product! And when I did, I found the following footnote (this is the complete reference with nothing changed) – “Based on an independent research study.” I guess the public is highly susceptible.
Here is another meaningless “statistic” – 95% of our customers are satisfied with … (fill in the blank). This time two questions come to mind:
- How large is the response pool?
- How is satisfaction defined?
As for question 1, I don’t care because I know it is a small and probably highly filtered sample with zero true meaning. But I always wonder about the second question.
The best practice in setting up survey questions is to have symmetrical choices. For example:
How satisfied are you with product X.”
- Extremely Satisfied
- Satisfied
- Neither Satisfied nor Dissatisfied
- Dissatisfied
- Extremely Dissatisfied
But many surveys are structured with a bias that distorts the “95% of our customers.” as follows:
- Extremely Satisfied
- Very Satisfied
- Satisfied
- Dissatisfied
- Extremely Dissatisfied
In the first example, the 95% satisfied selected the top 2 boxes. In the second example, they set the top 3 boxes. While I prefer the first example, it doesn’t matter if the person looking at the results knows how the number was calculated. But we never do!
And finally – % Top Box. The top box is generally considered a measure of customer delight and is the brass ring in the chase for customer satisfaction or loyalty. Yet, the literature references the top 1, 2, or 3 boxes. So, in the proceeding discussion, the top box means either extremely satisfied or extremely and very satisfied or the extremely plus very plus just plain happy. The last two are useful for marketing but not for managing your business.
Another example. When analyzing the “NPS” question, the 10s and 9’s are promoters, but, in my opinion, I’d rather have all 10s than all 9’s. So there is a difference in the importance of the top 2 boxes, and to manage an improvement program, I always recommend focusing on the 10s or any other pure top box.
How do you define the top box in your business? Do you relate satisfaction and loyalty data with business outcomes?
About Middlesex Consulting
Middlesex Consulting is an experienced team of professionals with the primary goal of helping capital equipment companies create more value for their clients and stakeholders. Middlesex Consulting continues to provide superior solutions to meet the needs of its clients by focusing on our strengths in Services, Manufacturing, Customer Experience, and Engineering. If you want to learn more about how we can help your organization collect statistically valid satisfaction and loyalty data, please get in touch with us or check out some of our free articles and white papers here.