Today, we won’t talk about strategy, objectives and improving ultimate business performance. Questions that I receive often indicate that sometimes it’s worth returning to the basics. Today, I suggest you pass a “Scorecard and KPI 101” course. “101” normally means an introductory level of learning, and in our case, it might be a little bit more advanced as I assume that my readers already have some developed background in the field of business scorecards and KPIs.
KPIs, Indicators, Metrics
From a business viewpoint it doesn’t make sense to call something “KPI” until a business context is defined. Still, in this article I’ll be using the “KPI” term as it is more popularized. The better idea would be to avoid the “KPI” term and just use “metric” or “indicator” instead. These aspects were discussed in detail in a previous article.
What problems are KPIs and a scorecard supposed to solve?
Indicators are numerical values that are linked to some kind of process. In simple words, their primary goal is to show a number that can give us an idea about the current performance of the process. We can group these indicators and calculate their aggregated values, and in this case, we are talking about a scorecard.
Here is the list of the areas where an indicator and a scorecard can be applied.
On the indicator level:
- Show the current value of an indicator.
- Introduce the performance function of an indicator, e.g., a higher value means higher performance or higher value means lower performance.
- Interpret the value of an indicator on a baseline/target scale, e.g. calculate the progress in improving the value of the indicator.
On the scorecard level:
- Group indicators into categories. Specify the relative importance of the indicator compared to other indicators.
- Calculate the performance of the category and the total performance by taking into account the performance values and weights of the indicators.
Let’s review these ideas from basics to the calculation of the total performance index.
The Value of an Indicator
Let’s start with simple examples of indicator values:
- $530 – “530” is a numeric value, and “$” is a measure unit.
- 20 hours/week – “20” is a numeric value, and “hours/week” is a measurement unit.
What if an indicator doesn’t have a numeric value, but we still want to use it? For example: when we deal with an expert opinion, one needs to come up with a way to quantify it. Instead of answering a question, the expert is supposed to choose one of the options associated with a numeric value.
The Role of the Performance Function
There is one important idea in the previous paragraph: each indicator has its own measurement units. On the scorecard, we cannot deal directly with incomparable measurement units like “$” and “hours/week.” But we can come up with a mathematical function that will put various indicators on the same scale. I’m talking about the performance function for an indicator.
What actually does the value “$530” mean? Is it a high value or low value? We cannot answer these questions until we have a measurement scale. Let’s create it:
- Min – means the minimal possible value of an indicator
- Max – means the maximal possible value of an indicator
- If min=$0 and max=$600, then we can say that an indicator with the value $530 actually tells us that we are doing well!
The performance of an indicator can be calculated as:
- Performance (Value), % = ((Value – Min) / (Max – Min)) * 100%
In our example, the performance will be (530 – 0) / (600-0) = 88%
The performance function is not always linear
The formula that was introduced above is a linear function. This means that with a linear growth of the “value,” the performance will also grow linearly. This is the most frequent case for a performance function, but there might be other performance functions as well.
For example, the performance might grow very slowly in the beginning but then increase rapidly. The performance function, in this case, might be something like this:
- Performance (Value), % = Power(Value,10) / Power(Max, 10)
Before, we were talking only about the performance that grows with the increase of the value, but it might be decreasing as well. For example, the more waiting time there is in the support center, the lower its performance. In this case the linear performance function will look like this:
- Performance (Value), % = ((Max – Value) / (Max – Min)) * 100%
This might sound a little bit complicated if you’re using MS Excel or similar software, but a professional scorecard software, like BSC Designer will automate these tasks for you.
Calculating the Progress
Sometimes, it is necessary to focus attention on a very specific part of the performance interval, which is important for the current business task.
For example, the support center of a company uses an “Average email response time” indicator. The Min can be 0, and the Max can be 72 hours. Currently, a company answers most questions within 48 hours. According to these numbers, the company performance is at a good level.
Let’s continue with a case: according to the information from the latest studies, “queries from online prospects that were answered within an hour were seven times more likely to generate a qualified lead.” Managers decided to decrease the average email response time. For their planning horizon, they have a starting point (baseline), which is 48 hours, and they have a destination point (target) which is 1 hour. But the performance function doesn’t take these values into account.
For this purpose, one could introduce a progress function, which will be very similar to the performance. The new function will actually use the same mathematical function as was used for the performance function, but it will use it on a different scale. Compare:
- Performance (Value), % = ((Max – Value) / (Max – Min)) * 100%
- Progress (Value), % = ((Value – Baseline) / (Target – Baseline)) * 100%
Both functions indicate that the performance will increase with the decrease of the response time. What will happen to the performance and progress when a company has a response time equal to 38 hours? Let’s calculate:
- Performance (38 hours), % = ((72 – 38) / (72 – 0)) * 100% = 47%
- Progress (38 hours), % = ((38 – 48) / (1 – 48)) * 100% = 21%
As you can see, we need both: performance and progress functions as they actually tell us a different story about an evaluation process:
- “Min” and “Max” are used to calculate the performance of the KPI. The performance answers the question: “What is the overall success according to the KPI?”
- “Baseline” and “Target” are used to calculate the progress. The progress answers the question: “To what extent was the target achieved?”
If you calculate the performance and the progress for the value= 1 hour (specified target was achieved), then it will be 100% for the progress (we need to define this point manually to avoid division by zero), and the performance will be 98%.
In terms of the “performance” there is space for a further 2% improvement, but in terms of the “progress” that reflects a business sense, the target was achieved.
Scorecard level. Weight – the Relative Importance of KPIs
Another challenge that was mentioned above is that we need to specify the relative importance of a KPI in some way. For example, a support center has two KPIs:
- “Average email response time,” hours
- “First contact resolution rate,” %
A company can make “Average email response time” equal to 10 minutes, but this will not make customers happy, as the “First contact resolution rate” will be very low. A good response time is important, but what is more important is the high quality of the answers. To reflect this idea, we need to introduce a concept of “weight” to the indicator.
- “Average email response time,” hours; weight = 4
- “First contact resolution rate,” %; weight = 6
Now we know that “First contact resolution rate” is more important than “Average email response time.” To simplify future calculations, we will define weight on a scale from 1 to 10, and we will require that the sum of all weights has to be 10.
Learn more about the business application of the “weight” idea.
If you use some software like BSC Designer for your business scorecard, then most likely it will suggest more flexible settings for the weight, and you will avoid unnecessary math exercises.
Calculating the Total Performance
We know the performance of each indicator in the scorecard. Also, we know the relative importance (the weight) of each indicator compared to the relative importance of other indicators in the category.
- The performance of a category can be calculated by taking into account the performance values of each indicator and their weights.
In the same way, the total performance of the scorecard can be calculated. It will incorporate the performance of all indicators taking into account their relative weights and the relative weights of their categories.
The Business Meaning of the “total index”
Below, you will find formulas that help to do all of these calculations. Before talking about mathematics, I’d like to discuss the business sense of calculating this total performance value.
Actually, it is still disputable. Some say that they need to have “a number” that is supposed to reflect the current performance of the company. An opposite argument is that this aggregated index will be too complex to possibly give any meaningful information.
It might be hard to find the business meaning of the total scorecard index, but for sure, it makes sense to calculate the performance of specific categories. As in the example above, the performance level of the “Customer support” category has two weighted indicators: “Average email response time” and “First contact resolution rate” that will actually show if customer support does a good job balancing response quality and response time.
How the Scorecard Performance is Calculated
Now, let me show the math that stands behind the calculation of the total performance.
Here we have the structure of the scorecard where C1..4 – are categories. Metric-i,j are our indicators together with their weight and performance values:
Let’s convert this into a different notation:
We had our weight on a scale 1..10, so before moving ahead, we need to calculate a normalized weight:
The total performance value for the selected category is to be calculated as:
Where Ni is the number of metrics on i-level; NWi,j – is a normalized weight of j-metric on i-level; NSi,j – is a performance of j-metric on i-level.
To calculate the total performance within all categories, it’s necessary to summarize performance values for all levels:
Where M is a number of categories. The final formula for the total performance index of the scorecard will be:
These calculations are simple if you don’t have subcategories in your scorecard. If you had subcategories, the weight of these subcategories should be taken into account in a similar way. Professional scorecard software like BSC Designer will automate these calculations so that you can focus on the business side.
Calculations for Leading and Lagging Indicators
Before, we have discussed the difference between leading and lagging indicators. Here, I’d like to focus on the topic of calculations.
To explain how the performance is calculated and transferred in this case, I will need to use an example. Let’s assume that we have a “Improve customer service” goal that is linked to “Create video tutorials” and “Train support agents” goals.
The “create video tutorials” goal is measured by:
- Leading indicator “Tutorials coverage, %” that will show the percent of the topics covered by the video tutorials.
- Another indicator is “Average watch time” – a basic indicator to estimate the engagement rate of the users who watch video tutorials.
The “train support agents” goal is measured by:
- Leading indicator “training time” that shows time invested, and another leading indicator “participation rate” that shows the coverage of the training.
- Lagging indicator in this case can be “evaluation test score” – it helps to get an idea about the effectiveness of the training.
We can introduce some random values and calculate leading and lagging performance for the goals.
- The leading performance will be calculated using leading indicators only, and respectively.
- The lagging performance will be calculated using lagging indicators aligned with the selected goal.
From the business point of view:
- Leading performance tells us a story about our efforts (if the trainer invested enough time in training and enough support agents have participated), while
- Lagging performance helps to validate our achievements in the context of this goal (did support agents improve in their evaluation tests?).
The goals “Create video tutorials” and “Train support agents” are linked to the “Improve customer service” goal and will transfer their performance up to “Improve customer service” goal.
Do these goals transfer leading performance, lagging performance or both? They only directly transfer lagging performance! In other words, the output of “Create video tutorials” and “Train support agents” become an input for “Improve customer service” goal.
What happened with their leading performance? Wasn’t it lost? As mentioned before, it helped us to validate our efforts, and if the hypothesis of the business goal (we are talking about the scientific approach) was correct, it will be converted into the lagging performance.
For example, for the goal “create video tutorials,” our hypothesis was that by explaining 80% of the product’s features in the video tutorials (as measured by leading metric “Tutorials coverage”), we will achieve a certain engagement rate (measured as “Average watch time”). If our hypothesis was correct, we would see the confirmation of this hypothesis on the dashboard diagram where both metrics are visualized. In this case, I see that lagging performance is growing simultaneously with leading performance:
Answering the question about leading performance, it is correct to say that leading performance is processed into the lagging performance in a case when the hypothesis that stands behind the goal proved to be true.
Getting back to our example. The leading performance of the goal, “Improve customer service” is now influenced by the performance of two goals, “Create video tutorials” and “Train support agents.” To measure lagging performance, we will need to come up with some indicators, for example “Retention rate, %.”
Examples of the scorecard in MS Excel
Here is an example of business scorecard in Excel. The calculations there work as described in this article. The general problem with scorecards in Excel is that when your project gets updated; it will be hard to maintain a spreadsheet. Check out an article on this topic if you are interested in the details.
What is your experience with scorecards? Do you think it is only useful as an academic exercise? What tasks do a scorecard help to solve in your business?