Balanced Scorecard and KPIs are powerful business tools, and they work not only for a business strategy, but for smaller things as well, like for example, for the software quality.
Classical KPIs like “Number of defects per line of code” or “Number of bugs per release” are not the best metrics to indicate the quality of your software. Read this article to learn what better alternatives are, and how to build excellent BSC and KPIs for software quality.
Using Balanced Scorecard to measure software quality
Let me ask you a question:
- What KPIs do you use to measure software quality right now?
- And the second question I’d like to ask is how do you actually USE them? I mean, how these KPIs help you to improve your business?
Here is the most important thing about Balanced Scorecard:
Control results that matter, not actions that might change from project to project.
How do you apply the Balanced Scorecard method to software quality issues?
Let’s have a look at software quality from the viewpoint of Balanced Scorecard. Remember the four perspectives of BSC that we discussed before? These perspectives will help us.
- Internal perspective: here is a place for software development and testing. Here we will put KPIs to estimate results in this area.
- Education and growth: how good is our team? How fast is it learning?
- Finance: How successful is our product from a financial view point?
- Customer: What do customers say about software quality?
Balanced Scorecard gives us this basic framework that we will use later to introduce specific measures.
What is software quality?
Now, let me ask you this. What do you think of when I say “software quality”? Here is what most users think about “software quality”:
Windows’ blue screen of death!
Now, let’s start brainstorming this idea. Why not introduce the measure:
“The number of blue screens”?!
However, let’s do what Balanced Scorecard experts normally do—rename this into something that sounds more scientific:
“Number of critical defects compared to the total number of defects”.
The first pitfall
Here is the first pitfall that I’ve mentioned before. We started measuring something just because we could.
- How are people going to use this KPI?
- Is it possible to define which bug is critical?
Any “blue screen” is critical even if it happens only once! I don’t think that this is the best KPI we could introduce.
What other measure could we introduce for “Software development and testing”?
Another commonly used KPI is:
“Percentage of unit tests converting software specification”.
Will this KPI help you if you have a stupid developer? Yes, your test coverage will be high, but what about general use experience?
- Don’t measure something just because you can.
- Don’t measure something if you have no idea how to link this to the final result.
What might be a better KPI in this case? I remember a good case about database re-factoring. My colleague was in consulting for a large company, and his goal was to make the structure of the database cleaner.
The first thing he suggested was:
- “Let’s rename column with name CID to customer_id to make code more readable in the future.”
The response of the management was:
- “Sorry, but it is too hard to do”.
If it is hard to change just one column name, how can this company think about improving the use experience of their product?!
KPI: How fast are you?
This is one of the best KPIs for the “Internal” perspective of the BSC:
How long do you need to deliver a new feature or to introduce a new change to software?
Using this KPI you will find possible communication and technical problems. With this KPI you will have room for improvement!
Metric: what is this for?
If you need another universal metric to estimate the quality of software from the view point of developers then use:
“What is this for?” per “Specification Page”
“What is this for?” per “Minute of Review”
As famous comic strip suggests that’s probably the only good way to differentiate bad code from good one. But be sure to validate the reviewer’s skills first before you start practicing this approach.
Less useful internal metrics
Let me introduce you to more typical and often useless KPIs:
- Number of bugs per release. Do you know any software without bugs? And what is the “normal” number of bugs you want to achieve? Zero I guess, but do you think it possible? Again, it is an example of measuring something just because we can.
- Number of defects per line of code. Do you see any real use for this KPI, other than telling you that you have bugs and need to fix them?
Don’t measure something just because you can!
So what is better to measure in this case?
Instead of measuring number of bugs, measure the number of returning bugs!
This KPI is easier to measure, and it will give you much more information about your team. For example, how fast your team is learning its lessons.
Now let’s talk about the Financial perspective.
- How successful our product is?
There are a lot of useful financial indicators. One I like especially is:
- “Sales growth when update is released”.
This KPI will give you a clue about the quality of your software product, but you will need historical data to understand the results better.
How do your customers react? Here it makes sense to return to the “critical bug”.
- KPI: What is the number of critical bug reports submitted by your customers?
- KPI: What is the number of requests for new features submitted by customers?
These two KPIs will give you a clue to improving your product and your processes.
Another idea that is not directly related to software quality is that you might use some service like socialmention.com to understand what your customers think about your product.
Software Quality Balanced Scorecard
Now, let’s have a look at the Balanced Scorecard that we have for software quality.
- In the “Education and Growth” perspective we want to understand how fast our company is learning from its mistakes, so we are measuring “Number of returning bugs”.
- In the “Internal Processes” perspective we have introduced the KPI “Time needed to deliver a new feature” that will help us to understand the direction of further optimization of business processes.
- Finally, in the “Finance” perspective we are free to use any relevant KPI, such as “Market share” or “Revenue”.
How do you start doing all these in practice? Try our BSC Designer software. There are several editions, including professional and freeware. Try it to automate as many parts of your Balanced Scorecard routine as possible. What I have just shown you is a basic example of where you might start.
Balanced Scorecard and KPI Checklist
You will need to use the same ideas to design your own Balanced Scorecards, so let me give you a ready-to-use BSC algorithm together with a KPI checklist.
Step-by-step algorithm to design a good Balanced Scorecard
- Start with your business goals—what do you want achieve in your business?
- Define solutions that you want to use to solve these business problems.
- Define the KPIs to measure the performance of these solutions.
- Group the KPIs into four categories.
- Really important: Involve your team or you will face greater problems with motivation in the future.
Here is more detailed Balanced Scorecard implementation algorithm.
Once you have your own KPIs, use this checklist to find out if you are on the right track:
- Check the number of categories and indicators—4/4 rule. There should be about four categories with about four indicators in each category
- Check the top-level categories—four default categories. If they are similar to what was introduced in the original BSC concept, these standard categories give a balanced overview of what should exist in your project.
- KPIs are result-oriented. You don’t measure something just because of you can; your KPIs are linked to specific business goals.
- Check the relevant importance of KPIs. The relevant weight of KPIs should be balanced.
- Indicators are measurable.
- Check if you have overload with details. KPIs are not the business itself; they are just a road map for your business.
- KPIs describe 90% of business. If they describe less, they will be ineffective.
Learn more about best practices for KPIs.
Typical problems that you might face
Once more, what are typical problems of performance measurement and management?
- Measuring something just because you can. Focus instead on result-oriented KPIs.
- Overloading scorecard with KPIs. Focus on two or three KPIs that matter.
- Motivation problems: it is hard to involve people in a project that doesn’t help the actual business.
- Is your scorecard still alive? Test your scorecard after 2-3 months.
Learn more about BSC Typical Pitfalls.