I read a lot about architects. I always thought the phrase “God is in the Details” was originated by German-born architect Lugwig Mies van der Rohe.  I was really surprised to learn it is generally attributed to author Gustave Flaubert.  In either case, I didn’t create it and it still says much if we listen.

But how does this apply to satisfaction and loyalty information? Well, professional researchers worry about statistical validity, margins of error and 95% confidence, especially when copywriters, editors and advertising people start throwing around generalities, which sound impressive but don’t do justice to the data.

How many of us remember seeing a TV commercial that opened with “4 out of 5 dentists would recommend Trident® gum for their patients who chew gum”?  Did you have the same mental image I had (and still have) of 5 people in dental gowns standing in a row with 4 holding a thumb up and the other a thumb down?  Well, that was the basis for an ongoing ad campaign.  And I must confess that I had to Google the first part of the quote to identify the product!  And when I did, I found the following footnote (this is the complete reference with nothing changed) – “Based on an independent research study.” I guess the public is highly gullible.

Here is another meaningless “statistic”  – 95% of our customers are satisfied with … (fill in the blank).  This time two questions come to mind:

  1. How large is the response pool?
  2. How is satisfied defined?

As for question 1, I really don’t care because I know it is a small and probably highly filtered sample, which has zero true meaning.  But I always wonder about the second question.

Best practice in setting up survey questions is to have the choices be symmetrical.  For example:

How satisfied are you with product X”

  • Extremely Satisfied
  • Satisfied
  • Neither Satisfied nor Dissatisfied
  • Dissatisfied
  • Extremely Dissatisfied

But many surveys are structured with a bias that distorts the “95% of our customers.” as follows:

  • Extremely Satisfied
  • Very Satisfied
  • Satisfied
  • Dissatisfied
  • Extremely Dissatisfied

In the first example, the 95% satisfied selected the top 2 boxes.  In the second example, they selected the top 3 boxes. While I prefer the first example, it doesn’t really matter as long as the person looking at the results knows how the number was calculated.  But we never do!

And finally – % Top Box.  Top box is generally thought of as a measure of customer delight and is the brass ring in the chase for customer satisfaction or customer loyalty.  Yet, the literature is full of references to top 1, 2 or 3 boxes.  So, in the proceeding discussion top box, means either extremely satisfied or extremely and very satisfied, or the extremely plus very plus just plain satisfied.  The last two are useful for marketing but not to manage your business.

Another example.  When analyzing the “NPS” question the 10’s and 9’s are promoters but, in my opinion, I’d rather have all 10’s than all 9’s.  So there is a difference in importance of the top 2 boxes and to manage an improvement program I always recommend focusing on the 10’s or any other pure top box.

How do you define top box in your business?