Net Promoter Score (NPS) is a metric introduced in 2003 by Frederick Reichheld in his popular Harvard Business Review article The One Number You Need to Grow.
NPS is quite popular among executives in multiple industries including high-tech. The biggest advantage of NPS is that it simplifies the objective of a business to creating more “Promoters” and less “Detractors”. This concept is much easier for employees to understand and act upon – compared to complex, hard-to-understand, impossible-to-act-upon metrics.
This Wikipedia article lists more pros and cons of NPS.
The question I’d like to raise today is:
Is there a similar “ONE number” that can be used to measure the performance of Product Management teams?
I believe the answer is “Yes” – read on for my thoughts…
The ONE Number to Measure Product Management Teams
Before I jump into the ONE number – here are some metrics I’ve seen used by some companies, and why I don’t like them.
- Product Line Revenue
- PM teams do not control the revenue. Even if they do their jobs extremely well, revenue depends on execution by Marketing & Sales teams – over which PM teams usually have little control, and minimal impact.
- Variations of “revenue-based” metrics often used include Market Share, Win/Loss %, etc. They suffer from the same negatives.
- P&L of Product Line
- I don’t like this due to the same reason as above – PM teams have even less control over Profit & Loss.
- NPS
- I like NPS better than the other two metrics above. That said – I’ve personally worked at companies where NPS was determined far more by the Customer Support organization than the PM organization.
I do not like any of these metrics because they measure the performance of a team (PM team) using metrics over which they have little control, and further only a vague idea of how to influence those metrics.
As they say: If you don’t know where you’re going, you’ll probably end up somewhere else! 
Product Satisfaction Score (PSS)
I believe Product Management teams can be measured quite effectively by the following metric:
- Product Satisfaction Score
- PSS focuses on customer satisfaction with just the product.
- It excludes other factors such as customer support experience, friendliness of sales teams, etc.
My theory behind PSS is that the primary goal of a Product Management team is to understand the wants/needs/desires of customers and to build products that meet them better than competition. PSS is an effective measure of this.
PSS offers the following advantages:
- Easy for product managers to understand.
- While PSS does have dependencies on other teams such as Engineering – most PM teams have significant impact on Engineering and hence can influence the outcome.
- You can measure PSS for each product – and apply it for product managers responsible for that product.
- As a result, product managers can easily chart the course to achieving higher PSS scores – they control it to a large degree.
- To use a baseball analogy – my theory is that we should measure a leadoff hitter by a metric such as his batting average or OBP – not by the Win/Loss record of the team, total runs scored by the team, or even his RBI.
I do understand that there is a whole bunch of scenarios where PSS may not make sense – just like NPS doesn’t make sense in a whole bunch of scenarios. But that doesn’t make it any less useful as a metric that can be successfully used in the vast majority of scenarios.
In a future post, I’ll share some practical ways to measure PSS & will further expand on this concept.
Like This Post? Get More by Email
3) Do not read this blog one sentence per day during your lunch break, while eating a
In my opinions, a good MBO should be linked to the company goals, but it should also be (easily) measurable, and should be within the control of the employee (product managers or otherwise). The product managers role is all organizations is a very highly leveraged/ matrix role. Between customers, sales, product development, …so many things that are not directly in their control. So any measure you put in will require high amounts of dependencies with others within the company and many factors that you don’t control. Therein lies the difficulty of setting the product managers MBO. Things like revenue, margins, win/ losses ratios, cost + time to on-board, cost to support the product, user growth, … are great measures at the company/ product level, but in my opinion they are not such great measures for product manager MBO. That is why, we were using fairly tactical things like a signed-off road-map, release x.x planning, release x.x launch, # of prospect cycles participated,….. as PM MBO.
I agree with the metrics @piplzchoice is using as these are most closely tied to product success in a market.
Looking at the bigger picture, how can we use a single metrics to measure product management teams in teams of process efficiency, effectiveness and improvement? Product success is one dimension that maps to the quality of the team, but are there other dimensions of measurement?
The best feature of any ONE measurement is simplicity to grasp, and I generally like NPS for this reason. However it takes 3 points of reference to establish your position while navigating in an open ocean (as an analogy for a market). We opted to measure a difference between customer expectation between expectation and customer experience with a product along 3 gradients: Functionality, Reliability and Support as measurements of Product Management teams performance.
@ Jim – Thanks for sharing your thoughts, excellent comments!
I agree with you on the twin goals – a metric should help us continuously improve, and also help us communicate success to execs and peers.
I believe PSS does well in both aspects:
* It’s easy to understand for product managers, and easy to visualize how to improve it and act upon improving it.
* It’s also easy to communicate to C-level execs – at least in my “quick and dirty survey” of 4 execs/friends. Most execs already understand NPS, so they grasp PSS easily and can see how PM team delivers that metric (to a far greater degree than other teams).
MSS sounds interesting – I’d like to know more about it. If there’s a post, can you share a link?
You said:
—
I had issues with Product Management using “gut instinct” versus market evidence and validation. There was a history of “not talking to customers and connecting with the market.”
—
I’ve heard similar thoughts from many execs – they felt their PM team was just using “gut instinct” vs. any systematic process, and there was no easy to measure their performance or ongoing improvement. I believe PSS can help with this.
Perhaps PSS/MSS (or derivatives thereof) will lead to a more systematic, practical way to:
* Measure performance of PM teams,
* Improve performance continuously, and
* Communicate the immense value good PM teams add to a company.
@ Ivan -
Great comments, thanks!
I do agree with you – whenever we try to use ONE metric, it’s not going to be all-inclusive. Unless that ONE metric is a complex equation that consists of 20+ variables!
My thoughts are as follows:
Most of us in Product Management know the following when it comes to positioning our *product*: It’s not a good idea to try to be everything to everybody.
I think the same should apply to positioning our *profession* – i.e. Product Management.
In my experience, most C-level execs know what exactly the Sales team delivers, and how to measure its performance. However, they don’t know what exactly the PM team delivers (MRDs, PRDS??) and how to measure its performance.
It’s true that the PM team can and does contribute to many things – such as your examples of cost savings and revenue generation.
My goal with PSS is to attempt to answer the question: What is the *primary* responsibility/deliverable of PM teams? Is there an easy way (like NPS) to measure it?
PSS is my best answer yet to this question – I do agree that it has flaws, omissions, etc.
Michael – great post! In considering the ONE measurement for product management, I’m not sure there is one. I agree with your PSS idea, but realize PM leaders should assess their teams and apply measurement to improve as well as find ways to communicate success to executives.
As an example, I’ve what I call the Market Satisfaction Score. The premise of MSS is defining a set of objectives (I don’t like the word MBO) and measuring how successful a product manager or product marketing manager discovers their market, connects with customers provides insights that are concisely articulated.
In this example, you can tell I had issues with Product Management using “gut instinct” versus market evidence and validation. There was a history of “not talking to customers and connecting with the market.”
If each product management leader is doing their job, they can define a set of measurements that will be unique to each product manager and stage some goals for the entire team. If you are a team of one, you need to set some realistic goals for yourself and then measure them.
Michael,
I agree with your sentiment about measuring PMs on factors that they have little or no control over, but the reality is that PMs are judged by their managers and others in their organizations based on those factors, whether that is an official KPI or not. Whether that is fair or not is a topic for another day.
I see part of the problem being the search for a single measure. Product Management is a complex discipline and trying to button-hole it into a single number ignores many of the factors that make PMs (and products) successful.
Also, by having a single measure, the Product Manager’s motivation is skewed, which will have the effect of blinding them from opportunities that don’t fit squarely with the metric. For example, how does the PSS (or NPS) measure cost savings gained by product decisions (which can impact the profit margin of a product) or the ability of the PM to aid in the closing of new business due to their domain or product expertise? Neither of these would be captured in the metric, but are important and valuable for the company.
Don’t get me wrong…I would like to see better quantification of Product Manager performance, both as a way to signal what’s important for the PM to focus on (read: translated into MBOs), as well as for Product team leaders to effectively evaluate their team’s performance and identify how best to steer the team to success.
I just don’t see that a single number can capture all of that.
Ivan Chalif
TheProductologist.com