Vendor Risk Management Scorecard: Evaluation Criteria for 2024

Learn how to create a vendor evaluation scorecard, automate assurance questionnaires, and detect areas of risk to the organization caused by a supplier.

Third-party vendor validation has become an integral part of cybersecurity, procurement, compliance, and supply chain strategies. Previously, we discussed general practices behind the evaluation scorecard; in this article, we will discuss the creation of a vendor evaluation scorecard, using vendor risk management as an example.

A template for a vendor risk scorecard in BSC Designer Source: Vendor Risk Management Scorecard.

We will demonstrate how to:

• Define evaluation criteria,
• Calculate the overall security score,
• Collect required evidence,
• Track security scores dynamically, and
• Align the results with other functional scorecards.

Define the Set of Evaluation Criteria

Follow current best practices to define the set of evaluation criteria for the vendor risk scorecard.

In 2024, this may include:
Existence of a formal cybersecurity program
Implementation of multi-factor authentication
Regular vulnerability testing
Adoption of the ‘Least Privilege’ practice
SOC 2 compliance of the data center
Data encryption in transit and at rest
Cybersecurity insurance
Intrusion prevention and detection systems
Cybersecurity awareness training
Data breaches reported
The number of data breaches reported

Depending on the type of criteria, it can be configured as:

• Binary – with possible states “yes” or “no.”
• Quantitative (for example, measured in %) or qualitative (natural choice or Likert scale).
• Criteria can be optimized for maximization (like % of employees who passed cybersecurity awareness training) or minimization (like the number of data breaches).

Learn more about best practices for managing evaluation scorecard.

In BSC Designer:

1. Switch to Strategy Workspace.
2. Navigate to New > New Scorecard > More templates…
3. Use the “Vendor Risk Management” scorecard template.

Assign Weight Depending on Risk Profiles

Weight the evaluation criteria according to the risk profile of the vendor.

For example:

• Vendors with access to sensitive information will have a high weight for criteria like “Cybersecurity Insurance” or “Data breaches reported,” while
• Vendors with no access to sensitive information will have a high weight for more common criteria like “Multi-factor Authentication” and implementation of the “Least Privilege” practice.

In BSC Designer:

• Select an evaluation criterion.
• Switch to the Performance tab.
• Adjust the relevant weight in the Weight property.

Build Hierarchy of Vendors

Group vendors into a hierarchy based on the vendor tier. Propagate evaluation criteria for each vendor.

In BSC Designer:

• Create groups using the “Add” button,
• Copy and paste a set of evaluation criteria into each group, and
• Rename the set name to match the vendor name.

Build and Distribute Questionnaires

Prepare and distribute security questionnaires to vendors, and later import the results back into the vendor risk management scorecard.

To prepare a questionnaire:

1. Open a scorecard.
2. Select a set of criteria.
3. Select Tools > Export data.
4. Check the checkboxes: “Export current item only” and “Include child items”.
5. Use the “Export as template” option.
6. Click “Next” to finalize the export.

Feel free to adapt the resulting template to your needs. For example, renaming the column “Value” to “Answer” and providing any relevant recommendations for the respondents to the questionnaire.

Initiate Evaluation of Vendors

Assess vendors based on the evaluation criteria to identify relevant vulnerabilities.

• Enter evaluation scores manually into the scorecard or
• Import them from an Excel spreadsheet for self-assessment questionary by the vendor.

Attach relevant evidence provided by the vendor, such as certifications and policies in practice.

In BSC Designer:

• Update scores manually via the Data tab.
• Use Tools > Export data to export evaluation criteria to Excel for the vendor’s self-assessment.
• Attach evidence to the evaluation criteria or to the mitigation plans articulated via the Initiatives dialog.

Vendor Risk Evaluation

We can use the vendor evaluation data to estimate the risk of cybersecurity breaches for the vendor.

In this case:

• The total score of the vendor evaluation will contribute to the Likelihood of the risk.
• The Impact of the risk can be estimated according to the role of the vendor in the supply chain.

To set this up in BSC Designer:

1. Click Add – Add Risk.
2. Click the Data Source button for the Likelihood indicator.
3. Change the formula to: 100-%[Vendor 1] (the higher the vendor’s progress according to the scorecard, the lower the likelihood of the risk).

Add the risk diagram to the dashboard to visualize the overall risk landscape:

Continuous Risk Monitoring

Track changes in vendor evaluation scores over time, such as changes in relevant certifications or the number of data breaches.

• Define the revision period for each evaluation criterion
• Monitor issues relevant to the evaluation criteria.
• Plan for the improvement of vendor evaluation scores.
• Plan for vendor offboarding.

A template for an evaluation scorecard in BSC Designer. Source: Evaluation Scorecard.

In BSC Designer:

• Setup Update Interval for an indicator via Values Editor
• Use Values Editor to assign scores to specific dates.
• Use Dynamic columns to see how the score changes over time.
• Use Initiatives to track data breaches and mitigation actions—update status and time span.
• Use comments for the score to track issues and Initiatives to map improvement plans.

Alignment with Strategy

Align the vendor risk assessment scorecard with other strategy and function scorecards, such as governance or compliance scorecards.

• Use the overall risk score of the vendor portfolio.
• Use the risk scores of specific vendors.
• Cross link initiatives from various scorecards.

A good example of the need for strategic alignment is AI. Even if your organization is not planning to implement AI technologies, it will most likely be affected by their use through third-party vendors and the supply chain. A vendor scorecard needs to be aligned with the AI governance functional scorecard.

In BSC Designer:

• Copy the Vendor score item and Paste it into the relevant scorecard.
• Select the “Link by data” or “Link by context” option.

Conclusions

In this article, we discussed the steps to create a vendor risk management scorecard:

1. Definition of evaluation criteria
2. Assigning weights depending on the risk profile of the vendor
3. Continuous risk monitoring
4. Alignment of vendor risk score with other scorecards

Learn more about more specific mechanics of evaluation scorecards.

More About Strategic Planning

Strategic Planning Process:

Examples of the Balanced Scorecard:

Strategy Maps:

Alexis Savkín

Alexis is a Senior Strategy Consultant and CEO at BSC Designer, with over 20 years of experience in strategic planning. Alexis developed the “5 Step Strategy Implementation System” that helps companies with the practical implementation of their strategies. He is a regular speaker at industry conferences and has published over 100 articles on strategy and performance management, including the book “10 Step KPI System”. His work is frequently cited in academic research.