Net Promoter Score (or NPS as it is also known) has been written and spoken about thousands of times over the last few years. Whilst some see it as the simplest (and cheapest) way to capture customer feedback, others dismiss it as far too simplistic with an inability to allow the organisation using it to understand what issues need to addressed.
Using the NPS scoring system as an analogy, I am neither a promoter nor a detractor of the methodology – although I would say the same thing for any customer feedback mechanism. As I will tell any business that asks me – every organisation is different – different customers; employees; business structure; issues; environment etc.. What each individual organisation needs to do to improve the customer experience will de dependent on its situation. This includes the adoption of a customer feedback programme.
NPS is a very effective tool to help a business understand customer loyalty – but it does not apply to every business. I have very successfully encouraged organisations to adopt it. These tend to be businesses looking to grow – the more likely one customer is to recommend the business to someone else, the greater the likelihood is that they will achieve the growth they are looking for.
NPS is obviously not the only customer feedback mechanism available. Customer Satisfaction is very commonly deployed, whilst ‘Customer Effort Score’ has become more prominent in the last couple of years. None is right, none is wrong – what is important is that an organisation deploys what is right for them. More often than not, I will recommend using two methods – one to galvanise support in the organisation, and one to provide a granular understanding of the priorities for improvement.
Sadly, NPS is often abused and misused – I would like to share with you a very current example. Yesterday, Mrs Golding unfortunately had to pay a visit to the Accident & Emergency department of our local hospital in Chester. Naomi’s injury was a long way from life threatening, but she was sent there by our General Practitioner. To cut a long story short, Naomi had a piece of ceramic tile stuck in her foot – a little like a splinter – and it needed to be removed as quickly as possible.
I dropped Naomi off at the hospital at 11:30am, and collected her at 15:45. We did not expect her visit to be swift!! In the hours Naomi was at the hospital, she observed and experienced a number of things – from the packed, not particularly clean waiting room, to friendly nurses and helpful doctors. By the time she hobbled out of the hospital, her objective had been met – her foot was sore, but the problem was solved.
Of all of the experiences we have today – especially in the UK – the one that we might least expect to be asked about is our experience at an NHS hospital. When Naomi looked at her phone later that day and saw a text message from an unknown number – she was intrigued. The unknown number turned out to be from the Countess of Chester Hospital – the image below is of the text message:
Instantly Naomi knew I would be interested in seeing it – she was right. My immediate thought was No No No!! Why on earth is an Accident and Emergency department of the only NHS hospital with an A&E department in Chester asking the NPS question??? What better example can be provided of an inappropriate use of the NPS methodology? Why on earth would anyone NOT recommend the A&E department in a city where there is only one??? Would anyone ask a paramedic to drive on to the hospital in the next city as they were told by a friend not to use the local one??
I think that the intent of the text message is correct – the method is completely wrong. I once attended a meeting where the now defunct NHS Direct shared with the group that their NPS score was close to 100% – this was yet another example of an organisation using the wrong method – NHS Direct was unique – you would never have NOT recommended it!!
I can tell you right now that the Countess of Chester Hospital will learn nothing from this survey – they will not have any idea that the waiting room was not clean, that certain members of staff were kind and polite, that the waiting times were satisfactory or not. The NPS score they receive will be extremely high. They may perceive as a result that the A&E department is doing a great job.
There is NO POINT capturing customer feedback unless that feedback enables your organisation to understand what the priorities for improvement are. If you want to get a great customer feedback score for the purposes of vanity, put your business in a monopoly situation and then ask the NPS question. The A&E department at the Countess of Chester Hospital would be much better served asking patients how satisfied they were with a small number of touchpoints during their visit – from waiting room, to triage assessment, to staff engagement etc.. If they carry on asking patients if they would recommend the unrecommendable, the result will be quite simple – a complete waste of money!
Hi Ian, my cousin had this experience recently being asked for NPS about an A&E admission, not being familiar with customer feedback methods she was incensed to be asked if she would recommend an A&E department to friends and family, stating that she would not wish any of her friends or family to have an A&E experience at all!
Totally agree that this is of no value, as you need the quant and qual elements of the feedback to address any problems i.e. what was your score (if that is an appropriate question to ask of the type of experience) and WHY? That would have picked up the issues around cleanliness, friendliness etc.
Thanks for your feedback Charlotte – it does make you wonder whether executives of NHS trusts have ever sat back and considered how inappropriate the question really is!
Hi Ian.
It’s not the Trust’s fault in this instance. This is a nationally mandated exercise that is called the Friends & Family Test (FFT) which is compulsory for the Trusts to operate. Further than that, they are incentivised to achieve a certain percentage response rate (typically 15%).
Like you say, the intent is correct, as it is meant to drive a ‘customer experience’ ethos that may not be culturally embedded in the NHS. However, the critical point shouldn’t be the metric (the quantitative) but rather it should be the follow-up question of ‘Why did you give that score?’ that should provide the qualitative measure and to guide the Trust as to where they should focus their improvement. Having the former without the latter is missing the point. (As Charlotte says above).
The answer to the WHY? question gives the context, in the patients’ own words, and allows the Trust to take action on the insight they extract from that verbatim. If your wife was pregnant she would have been asked the same ‘NPS’ question at several different points in her journey so Trusts do try to pinpoint certain services, processes and locations (wards) but the key has to be WHY the patient gave that score in order for the NHS to learn.
Happy to provide more insight if you’d like to know more about the process.
Many thanks for taking the time to clarify – it is much appreciated. This is a great demonstration of a ‘one size fits all’ approach not being appropriate. Incentivising the adoption of an inappropriate measurement system will not lead to the effective insight they are looking for. I do believe that more creativity needs to be used when implementing feedback mechanisms – including the adaptation of wording to suit the scenario – in the case of an A&E department, recommendation simply does not work.
Hi Ian, I feel compelled to leave a comment, although I fear I’m not going to be able to articulate my sentiment fully, it’s more of a musing for us to perhaps ponder on.
I agree that the NHS ought to listen to feedback from its service users. It’s important to understand the experience had by those who have accessed its services in order to make improvements and to sense check whether the services are being delivered in an appropriate way.
However, I whole heartedly agree that an NPS survey is not appropriate for an A&E dept. I can’t help but feel that some bright spark in Central Govt has come up with a policy and it’s been rolled out with a disconnect in understanding between operational delivery and policy development – something that will be only too familiar to those in an operational/’grass roots’ role.
It riles me that Govt Depts continually mandate that we think of everyone as a ‘customer’ – you with your CXP hat on, might disagree. However, I would argue that there is a difference between a customer and a service user. A patient at an A&E dept is not a customer, but a service user. As you point out there is little choice in A&E depts, if your leg is hanging off, you’re unlikely to request a detour to your preferred hospital! I suspect this is part of a drive to allow patients choice in their treatment. For example, being able to select which specialist you’re referred to. Essentially having Trusts in competition with each other and ‘league tables’ seems a bit barking to me. Encouraging a competitive environment in a publicly funded arena doesn’t necessarily feel right – although I need to have a proper think about this as it’s a complex issue and I understand the need for VFM and efficiency, etc blah, blah, blah.
However, I digress. A passenger flying through an airport and passing through the Immigration Control is a customer of the airline, but is not a customer of the Immigration Service (or Border Control or whatever they’ve called themselves these days), similarly, a victim of crime is not a customer of the police, nor was your wife really a customer of the NHS. They are service users and the services need to think about what it is they’re trying to capture/improve/deliver. This does not mean that we should devalue the service user, they just feel different. Here’s where I struggle to articulate why, perhaps it’s about choice or the traditional scenario of money changing hands, I’m not sure. We can’t apply the same ‘customer is always right’, pander to needs and give a warm fuzzy feeling to service users that we classically associate with customers. Whilst providers of services should absolutely consider the needs and feelings of its users, I’m not sure it’s paramount to the delivery of the actual service. My feeling is that access these essential services ought to be easy and so the Effort metric would make far more sense. There is also the more emotional feedback about interactions with representatives of that service – did you feel confident that they gave the best treatment/advice/action etc. Of course we should all be polite and considerate, but I’m not going to suggest to a surgeon that I’d like to have my right toe cut off instead of my left one please, the same way that I would ask a salesman for a blue car instead of a red one, there is far less choice involved and so it’s a different experience. The generic NPS question does not fit.
This shouldn’t be a ‘recommend’ question, they’re missing an opportunity to gather some really interesting, appropriate ‘customer’ or service user feedback. And I have to wonder, and ask the obvious question: How much is it costing our beleaguered NHS to make this obvious mistake and that’s a shame.
I’m replying to my own comments now – never a good sign!
I guess what I’m trying to say in summary, or at least ask is:
Is there a different metric for services as a whole? Does ‘recommend’ only work where there is choice, desire and a certain amount of kudos or associated with an actual purchase?
Should a ‘service’ use a metric that reflects what it’s really about? For me it’s access, ease of use, reassurance and confidence. Not recommendation.
Hi; we have looked into conducting the F&F test for a couple of Trusts and also a national chain of NHS Dentists who also have to undertake the measurement as do Dr surgeries. I agree with everything that has been said but the thing that perturbs me more than anything else – especially as my mother just had a major operation at a huge County Hospital at the 4th time of asking! – is the money it’s costing the NHS. I have seen tenders released on various sites from trusts looking for agencies to undertake the measurement (which is usually a pen and paper exercise so actually impressed by the Chester example) and the budgets are eye watering! I dread to think how many more nurses, paramedics, ambulances, beds or junior Doctors could be funded with that money! Someone within central government needs to take a serious look at the costs involved as well as the efficacy, reliability and validity of the measure. Consultancy with the industry would be a great starting point and if Government insist on measurement they should fund it centrally and leverage their buying power – we all know the NHS are woeful at that particular exercise.
Do not fear Kirsty – this is a brilliantly worded assessment of the situation – in your follow up comment I think you summarise the situation perfectly. Organisations should not obsess about using the word ‘customer’ if the user of the product or service has no choice. One thing that organisations like this should obsess about is the ‘outcome’ that the user of the product or service is expecting and whether or not the outcome was successfully achieved. In understanding this and ‘why’ the user responded in the way they did, a far more practical understanding of the priorities to improve will be obtained. I love your view of access, ease of use, reassurance and confidence – would make a lovely follow up guest blog post if I could convince you to write it 😉
Great blog to highlight a growing trend of surveys for surveys sake.
I read recently about Goodhart’s Law which seems to apply here, especially when you consider the governments intentions around measurement of NHS services.
Goodhart’s law states “As soon as the government attempts to regulate any particular set of financial assets, these become unreliable as indicators of economic trends”, or more simply when a measure becomes a target, it ceases to be a good measure.
In this case the NPS is the focus and is meaningless and irrelevant without qualitive feedback.
Fantastic – thanks again for sharing Jase (I have not come across this before) – sums it up perfectly!
Hi Ian – have you seen this? According to the Independent “Hip replacement operations more popular than iPhones, according to NHS data” – http://ow.ly/zxo58 *shakes head*
I had not Jase – many thanks for sharing – I too shake my head vehemently! The metric is so dangerous when used out of context. Comparing a knee replacement with an iphone – a metric designed to measure loyalty. If the government wants to use a measure to massage their egos, then they have chosen the right one.
[…] Originally published at: ijgolding.com […]
Great article Ian! Recently we published an e-book on how to ask customers the ‘right’ questions for VoC insights.You can find it here: http://get.usabilitytools.com/49-questions/ I’d be more than happy to get your feedback after your read it!
Hi Ian – Ghulam here least I have not yet subscribed to your blog. A great post. Helps one put together a great use case for notnusing nps. But just to give the NHS a break ans since they are regulated to ask the question i would have them at least reword it to “if you’re friend was deciding to move to chester based on the quality of service provided by A&E would you recommend them to move here?” LoL
Getting to this rather late, but I’ve become increasingly frustrated with the use and abuse of NP. It seems more often than not to be used as an excuse not to think.
However, I think pointing out that NPS is meaningless in the context of monopoly doesn’t really do the problem justice. People don’t think rationally about whether they are considering something that has an alternative or not. They do, however, think (when asked by NPS) about whether they could see themselves recommending an experience to a friend or relative. This is opposed to the rather more straightforward consideration of much they like or dislike something.
Considering whether you like something is very different from considering whether you would recommend it. The latter has secondary considerations around it, not least if you don’t habitually recommend things anyway. If NPS is a proxy for like/dislike, then it’s taking on a lot of rather random baggage that isn’t in my view properly considered.