Bad KPIs: Why Does Misuse Happen and How Avoid It

As with any other business tool, Key Performance Indicators have disadvantages, one of them is a possible misuse of KPIs. You set up some good metrics, but soon you find out that your employees have found a way to game the system. Instead of doing what you expect them to do, they have found a loophole to get KPIs into the green zone without doing what your business actually needs. 
How to avoid bad KPIs and KPIs Misuse?

Gaming KPIs is a Big Problem

The recent case with Wells Fargo where at least 5,300 employees were involved in opening sham accounts and getting credit for meeting their sales goals just confirms how big this problem might be.

  • You might find a call center where employees are calling each other just to keep their “Number of calls” indicator in the green zone.
  • A web marketing specialist could drive traffic to a website without caring much about its quality.
  • In the Soviet Union, the effectiveness of their railway system was measured using a “ton * kilometer” metric. And the simplest way to keep this indicator in the green zone was to move shipping containers across the country without any particular need.

In his book “The Tyranny of Metrics”1, Professor Jerry Z. Muller calls the current approach of many organizations to tracking performance – “metric fixation.” He shares more vivid examples of how even non-monetary rewards can lead to undesired outcomes:

  • Surgeons who don’t take difficult cases that might affect their public rating
  • Police officers who are downgrading crime classification

The list goes on, and if you know some good examples, feel free to share them in the comments.

Why do KPI Failures Like This Happen?

I’m sure there are some cases that happen because of the bad side of human nature, but in the majority of cases we need to blame management. The typical reasons are:

  • Pushing employees too hard towards working for KPIs (KPI itself becomes a target)
  • Linking incentives to the KPIs and not validating them later
  • Inventing KPIs on the top and mandating them to the bottom, skipping the discussion phase
  • Setting the target value for an indicator, but not discussing an action plan
  • Focusing on the destination, but forgetting about quality/value aspect

To solve misuse problems, we need to address both technical and cultural issues. A technical one will work effectively only if the performance measurement culture is updated as well.

Technical Solution: Use “Performance-Value” or “Performance-Quality” Pair

The technical solution is to use a pair of indicators, not just a single metric. This pair should be formed by:

  • Performance Indicator, and
  • Quality or Value Indicator.

For example, for the sales agents we might use:

  • Performance indicators: Accounts created
  • Quality indicator: Average account’s profit within a certain period of time

If a bank would use a similar pair of indicators instead of just one indicator, they would certainly be able to diagnose the problem of sham accounts at an early stage.

For the customer service specialist this might be:

  • Performance indicator: First call resolution rate
  • Value indicators: Returning problem rate; Net Promoter Score.

Is it possible to game a pair of indicators? It depends on the pair, but it is still possible. That’s why we need to address the problem on the cultural level as well.

  • If plan a long term change, I do recommend for you to check out this article about changing your performance measurement culture.

Cultural Solution: KPIs Are All About Discussion and Learning

Start with finding and eliminating bad behavior patterns:

  • Mandating KPIs top to bottom
  • Abusing KPIs in the command and control chain (just remember that KPI is the INDICATOR for the performance, not the PERFORMANCE)

Practice shows that KPIs work much better when:

  • They are the product of the discussion between the line-level employees and their managers
  • The KPI data is used as a base for another discussion, but not to blame someone for the bad results

KPIs and Incentives

I know many companies incorporate KPIs into their incentive plan. Everything looks logical at the first glance: there are bonuses that we want to pay, and we pay them only when a person hits certain targets. The thing is that by doing this a company is shifting a focus from a KPI as a learning tool (which it should be in a perfect case). In the previous article I shared my thoughts on the topic and discussed some best practices.

The Most Important Rule For the KPIs

Stop chasing KPIs: Focus on the value creation and the metrics that can help you to track the value.

What’s the Difference Between a Bad KPI and a Good One?

Good KPIs doesn’t come along; you need to have these parts:

  • A clear business goal behind all KPIs;
  • Performance driver indicator or leading indicator;
  • Result indicator;
  • Indicator related to the quality or to the value created;
  • An action plan that is aligned with indicators.

Good KPIs are normally accompanied by all these details, while the bad ones are not. It’s not rocket science, but you need to follow certain steps when brainstorming your KPIs. Feel free to use our KPI 10-step system to find good KPIs for your business.

An Example: Discussion Around KPIs

Let’s take online marketing as an example. This is a kind of discussion that might happen between a marketing specialist and his/her manager:

  • Manager: We need to increase website traffic by 5%! {That’s where many managers stop their management efforts}
  • Marketing: What about using a content marketing approach?
  • Manager: Sounds great. How you are going to get it to work?
  • Marketing: We will start writing interesting articles and posting them on our website and on the social media. {Here is an action plan!}
  • Manager: The more articles you write the more visitors we have, right?
  • Marketing: It’s not that simple these days. Search engines care about people who search information online, so we need to make sure those articles are high quality ones, and address real users’ needs. {We know what performance indicator will be used – The number of articles, but it is not enough}
  • Manager: How can we make sure that these are high quality articles?
  • Marketing: We’ll need to study first the problems that our users deal with and find an expert on the topic who will be able to write about them. {This is where the value is created}
  • Manager: Once everything is done, how can we validate the results?
  • Marketing: We can track the number of visitors that were attracted by the new content!
  • Manager: And how do we know that those are our target users and that we created any value for them?
  • Marketing: We can track the social engagement metrics, time they spend reading the article, and track the conversion rate into our mail list and later into the sales. {That’s where we validate value created and quality achieved}
  • Manager: Now we have a complete picture! {This is not the end of the dialog, after a certain period the team will meet again to analyze the results achieved}

Here are the results of the discussion:

  • Business goal: attracting more qualified visitors to the website
  • Performance driver indicator: the number of articles published;
  • Quality/value indicators: conversion into mail list/sales; time on site; social engagement metrics;
  • Action plan: 1. Study customer’s problems; 2. Find an expert. 3. Write and publish content; 4. Analyze the results and improve.

I think readers would agree with two points:

  • For both parts, manager and the marketing guy, this approach to the goal/KPI works much better than just “Increase our website traffic by 5%.”
  • There is no sense in a marketing employee gaming the system and, for example, writing many poor-quality articles. That simply won’t work and the quality/value indicators will show that something is wrong.

Automation Tools

If you want to follow this model, then instead of 2 columns with an indicator name and its value, your spreadsheet must be something more detailed:

  • First of all it is necessary to show a business context, preferably a strategy map;
  • Then more indicators need to be tracked, as well as an action plan;
  • And, you also need to keep a historical record of all performance data, as well as have a reminder to update and check indicators’ value regularly.

Retail Banking Balanced Scorecard Strategy Map

I’m sure Excel gurus can solve these tasks very well; the problem might come when someone else needs to update this spreadsheet…

An alternative way is using a professional software for business scorecards, like BSC Designer. There you will be able to:

  • Store a number of strategy maps to explain business context
  • Add various indicators and align them with your business goals
  • Assign a person responsible for an indicator
  • Track budget, initiatives, action plans related to the business goals
  • Specify preferred update interval for the indicators and make sure that the values were actually updated on time
  • Once it’s time to analyze the results, you can visualize all the data on the dashboards or in the performance reports

Give it a try! If you don’t like the idea, you can always export your data back into an Excel spreadsheet.

Vanity Metrics

Another typical bad practice in the performance measurement domain is the vanity metrics.

Vanity metrics are easy to be proud of, but they have low correlation with desired results.

Why vanity metrics are a problem:

A sign of having a scorecard full of vanity metrics is when the combination of leading/lagging indicators is not balanced. For example, the “training hours” indicator, combined with “change in behaviour, %” indicator is a great way to track the training in the organization. However, if we focus just on “training hours,” we are giving a talent manager a vanity metric that is easy to manipulate. 40 hours of training per employee looks impressive on the corporate dashboard, but where are the tangible results of this training?

Let’s look at some typical examples of vanity metrics and their more valuable alternatives.

Vanity Metric Better Alternative
Website visitors – easily manipulated with junk traffic or online ads. Qualified leads – are much better proxies of value. For example, at BSC Designer, we measure the number of opened accounts where users tried some features.
% of Trained Employees, Training budget, Training time. Training effectiveness (for example, according to Kirkpatrick’s model).
Lines of software code – easy to calculate but hardly correlates with quality of the code. The % of returning bugs – better correlates with product quality.

Conclusions

Here some key take-aways of the article:

  • KPI misuse is a big issue; any business is at risk
  • Focus your team on the value creation, not on the KPIs and respective targets
  • Be careful with incentives linked to the KPIs, they are shifting focus from learning
  • Make sure your performance KPI is paired with a value/quality indicator
  • The best KPIs are the product of the discussion
  • Prepare your culture for KPIs in the red zone: the best insights come from there
  1. The Tyranny of Metrics, Jerry Z Muller, Princeton University Press, 2018
Cite as: Alexis Savkín, "Bad KPIs: Why Does Misuse Happen and How Avoid It," BSC Designer, October 18, 2016, https://bscdesigner.com/avoiding-kpis-misuse.htm.

1 thought on “Bad KPIs: Why Does Misuse Happen and How Avoid It”

  1. Another case of KPI misuse and possible prevention plans for it

    Shipment service is trying to improve their service by introducing an “on time delivery” indicator. Chasing for that metric, truck drivers, produce fake delivery reports (they report that delivery failed because of the absence of the recipient). On the dashboard everything looks perfect, but actual customer service quality is at risk.

    A possible solution to this problem would be to create a quality metric. Give clients a simple way to share their feedback about the delivery service, and use this data to validate the “on time delivery” indicator.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.