Compare the performance of metrics over time. Support data-driven decisions with analysis of trends, variance, correlation, and benchmarks.
In strategic planning, we quantify objectives to make them more specific. The absolute value of a metric typically provides limited information. For data-driven decisions, collect data for the metric and use tools such as:
- Trends and anomaly detection
- Variance analysis
- Comparative period analysis
- Correlation analysis
- Internal and external benchmarks
In this article, we’ll discuss best practices for using these analytical tools and share some examples for users of BSC Designer software.
Track Historical Data to Enable Trend Analysis and Anomaly Detection
The starting point for metrics analysis is to gather data over time. To ensure consistent measurement:
- Define the actualization interval
- Define the measurement method
- Assign a person responsible for revising the indicator
If the data is already available in an IT system, consider setting up automatic updates.
When gathering historical data, ensure the person responsible can note down relevant comments and insights, such as: “Sales were low this month because the office was affected by an extreme weather event.”
Availability of historical data enables the use of other analytical tools such as:
- Trend analysis
- Anomaly detection
- Benchmarking
It’s also a prerequisite for any AI-based analysis.
In BSC Designer:
- Assign a person responsible via the Owner field
- Set the update interval for the indicator via the Values Editor
- Enter data manually or automatically
Metric Analytics: Practical Examples. - Visualize historical data on the dashboard as a data table or time chart
- Enable a trend line for visualization on the chart
Metric Analytics: Practical Examples. Variance Analysis (Actual vs. Expected)
In strategic planning, we typically focus on improvement goals. From a performance measurement perspective, we expect the current state of a metric to change, ideally reflecting improvement.
To implement this, we define a baseline and target for the metric, creating a performance measurement scale. The current value is then analyzed on this scale, which is also called normalization.
Once all metrics on the scorecard are normalized, they become comparable. For example, the absolute sales numbers of a small regional office and an office based in a financial hub might not be comparable, but by normalizing the sales data using targets defined for each office, we make them comparable. We’ll use normalization later when discussing benchmarking analysis.
In BSC Designer:
- Enter the current value, baseline, and target on the Data tab
- Specify the optimization formula on the Performance tab
Metric Analytics: Practical Examples. The tool will calculate the progress for the indicator.
In some cases, two scales for normalization are needed. In BSC Designer, switch to the Data tab and disable “Simple mode” to activate the additional “Min” and “Max” fields. This gives two scales for normalization: min-max and baseline-target. The tool will calculate both “performance” and “progress.”
Comparative Period Analysis: Month-over-Month, Year-over-Year
Another approach to analyzing historical data is comparative period analysis, which helps to:
- Detect trends
- Track change rates
- Understand seasonal effects
Instead of looking at discrete data records, group data by periods, such as months, quarters, or years, to track changes over time.
For example, when analyzing website traffic month-by-month, trends might not be visible, but grouping data by years may reveal positive or negative trends.
The method for data grouping depends on the nature of the data and the context of measurement:
- Metrics like “customer satisfaction rate” need to be averaged
- Metrics like “monthly sales” need to be summarized
If you are interested in detecting anomalies, the grouping method might be changed to the:
- Minimum value, or
- Maximum value
In BSC Designer:
- Set up the grouping type of an indicator via the “group by” control in the Values Editor
- Display the “Dynamic” column in KPI tables, reports, or on the strategy map
- Use “group by” controls on the KPIs tab, Dashboard tab, and in the Reports dialog to change grouping periods for visualised data
Metric Analytics: Practical Examples. Correlation Analysis with AI
Once you have historical data for various indicators, you can move on to correlation analysis. It can be done manually by reviewing the data or using AI to identify possible correlations between indicators.
For manual analysis, visualize two or more metrics on the same chart. In BSC Designer, you can do this by selecting several indicators as a data source.
Metric Analytics: Practical Examples. For AI-powered analysis, provide AI with the contextual information and data for the metrics, and ask it to identify possible correlations.
Make sure to apply critical thinking to the results of this analysis, as correlation does not imply causation.
To use correlation analysis in BSC Designer:
- Switch to the AI tab
- Start a new chat by providing performance data to AI
- Ask it to find KPIs that might correlate
Here is an example of prompt for AI:
You are an AI data analyst tasked with performing a comprehensive correlation analysis on a Balanced Scorecard containing various key performance indicators (KPIs) across multiple perspectives. The goal is to understand the relationships between these indicators to inform strategic decision-making.
Instructions:
– Focus on the data span of last year
– Compute the Pearson correlation coefficient for each pair of metrics.
– If data is not normally distributed, use Spearman’s rank correlation coefficient.
– Determine the statistical significance of each correlation coefficient.
– Use a significance level of 0.05 (95% confidence interval)
– Highlight pairs of indicators with correlation coefficients above 0.7 or below -0.7
– Provide a detailed interpretation of the significant correlations identified and possible reasons for these relationships based on business context.
– Propose hypotheses that could be tested with further analysis or experimentation.
Metric Analytics: Practical Examples. Benchmarking or Relative Performance Analysis
Benchmarking is useful when the same metric is used across a business domain. There might be:
- Industry-standard or best-practice metrics (e.g., “Gross profit margin” or “Net Promoter Score”)
- Metrics used to evaluate competition (e.g., “Market share” or “Website traffic”)
- Internal metrics used across departments (e.g., metrics in evaluation or vendor scorecards)
From a measurement standpoint, the key success factor in relative performance analysis is defining and maintaining measurement standards. While it’s realistic for internal metrics, for external metrics we need to accept a higher margin of error. Even classical metrics like “% of customers who recommended our product” can vary significantly across companies depending on the context of the question.
In BSC Designer:
- Create a template set of KPIs or evaluation criteria
- Propagate (copy and paste) the KPIs to represent various internal departments or competitors
- Use the “Series” chart on the dashboard to visualize data by evaluation criteria and compare the total performance of each benchmark
Metric Analytics: Practical Examples. We’ve discussed more specific examples and best practices for measuring data series in our discussions on evaluation scorecards.
Formulating Actionable Insights
The primary goal of performance data analysis is to discover insights that can be used to formulate better strategic hypotheses.
General principles for formulating these insights:
- Refer to the data that triggered the insight
- Verify sources and analyze the context, as any data can be unintentionally biased
- Conduct root cause analysis to better understand the insight
- Align insights with the existing strategy to support objectives, address stakeholders’ needs, mitigate risks, and more.
- Remember, an insight is a hypothesis that needs validation before scaling. Treat it accordingly by identifying relevant leading and lagging metrics and establishing experiments.
In BSC Designer:
- In the early stages, note down insights as comments for specific dates of selected KPIs
- At later stages, move insights into functional or strategy scorecards to develop them further
Metric Analytics: Practical Examples. Summary of Performance Analysis in Strategic Planning
- Defining performance metrics in the context of the formulated strategy and stakeholder needs.
- Tracking performance data for indicators over time.
- Using analytical tools discussed above to generate insights.
- Formulating new hypotheses and actionable insights.
- Validating hypotheses in practice; updating strategy with new inputs.
Alexis is a Senior Strategy Consultant and CEO at BSC Designer, with over 20 years of experience in strategic planning. Alexis developed the “5 Step Strategy Implementation System” that helps companies with the practical implementation of their strategies. He is a regular speaker at industry conferences and has published over 100 articles on strategy and performance management, including the book “10 Step KPI System”. His work is frequently cited in academic research.