Avoiding the Pitfalls of Nutty Net Promoter Scores

We have always been big fans of the Net Promoter Score (NPS) metric because it has convinced many firms to begin using customer satisfaction measurement scales that work better and that are tied to what people do rather than what people think.  Eleven point scales (with points zero to ten) allow for optimal variation.  They are intuitive and appealing: people quickly grasp the idea of rating something on a zero to ten scale, and are familiar with the idea from grade school.  They also have a neutral mid-point, which is important for many customer satisfaction and loyalty studies.

But NPS questions do not make sense in many situations.  Here’s one we saw last week—it’s a survey sent by Amazon to sellers who call regarding complicated issues with how their products are being displayed on the website or how payments are being transferred:

A Poor Use of the Net Promoter Question

The problem with this question is that any honest respondent will almost certainly answer “very unlikely” and give a score of zero.  The support representatives are anonymous, so it is impossible to recommend them.  If you call or email Amazon, you cannot request a specific representative, making the issue moot.  And even if you could recommend a specific representative, how often would you have the opportunity to make such a recommendation?  Few sellers talk to other sellers about the display and payment problems they are having.  So all in all, how likely do you think most sellers are to recommend the specific person they talked with?  If we have any faith in the reliability and validity of survey data, it should be about zero.

The hapless support representative likely got a lousy score (there goes her bonus!) not because she did a poor job, but because whoever is in charge of this tracking survey didn’t think very hard about NPS, how to use it and when to use it.  More generally, somebody didn’t think very hard about how to do smart research.

Staying on top of recent innovations, important industry trends, and best practices is critical.  We can help you with that, with suggestions about the right ways to apply them so that you get valid insights.  Your customers and your service reps will thank you.  And the next time your manager and executive team rate you, they will likely recommend you for the next important research project or satisfaction tracking study (score 9 or 10).

—Joe Hopper, Ph.D.