Customer satisfaction, employee engagement, and product quality – all can be measured using surveys. In this article, we discuss 8 ways a business professional can improve their survey techniques to get more business insights with less efforts. This article is also available in Spanish.
Here are the topics that we’ll discuss:
- What are the advantages and disadvantages of the surveys
- What are the alternatives
- Why bad business practices look great through the survey prism
- The problem of multiple choice questions and how to fix it
- Survey checklist with recommended best practices
- Some advice on how to present survey data on strategy scorecard
A quick overview of the surveys: advantages and disadvantages
Surveys are a great business tool. Many organizations use them to get some feedback from their customers.
Surveys help to cut costs of market analysis. Instead of thinking about sophisticated IT to track customer behaviour, an organization can simply ask their questions directly to the end users of their product.
The opposite side of the coin is that:
- The results of the surveys are subjective
- The data obtained is affected by observation bias (the way we formulate the question predetermines the way people will answer it)
- The answers to the pre-defined questions ignore important contextual nuances
- The free-form answers are hard to quantify
Probably the most dangerous drawback of the surveys is that their results can be easily manipulated. I’m not talking about an angry client who decided to give a “bad” rating to the customer service. I’m talking about the management team that unconsciously manipulates a survey on the early design stage by using multiple choice questions. Below we will talk in detail about this problem and how to avoid it.
Observation is an alternative to a survey
Do we have an alternative to the surveys? I discussed this in 10-Step KPI System book, the alternative is observation. Instead of interacting with the customers by asking questions, we could do what astrophysicists do with stars – observe without direct contact with the subject of observation.
Should we stop using surveys at all? No way! We need to understand the limitations of this tool, and learn to get the best possible results from using it.
Why bad business practices sometimes look great through the survey prism
On a late summer evening, you land at Charles de Gaulle Airport, France, and find out that your bank cards were blocked. After talking to a bank representative, you know that there was an IT error on their side, and that you won’t have any access to money on your bank account for at least one week until new cards can be mailed to you. They know your situation, and they care a lot, but instead of sending new cards urgently via FedEx directly to the hotel, they simply say “sorry” and promise to fix it as soon as possible. This is a bad business practice.
Some days later you get a call from the customer loyalty service of the bank. They want to know your opinion about the service provided to you recently. You are willing to let them know everything you think about their bank and explain how hard it was to survive without cards, but the format of the call implies that you answer the survey questions.
Let’s have a look at those questions:
- “Was the call answered fast?”.. could be better, but yes, it was.
- “How professional was the person who handled the call?” .. the bank representative was really professional; she responded calmly and explained in detail the reasons for the problem; she did not have a solution, but it obviously was not her fault, but the fault of the bank.
- “Would you recommend our bank to others?” The way bank managed the situation was not acceptable, but there are of course some benefits of being their client. Should I answer here “Never again” or try to be more realistic and say, “Probably yes…?!”
You are willing to add some more comments, but on the other end of the line all fields of the survey are now completed and you hear:
- “Thank you for your opinion. It is VERY important for us. Good bye.”
How will the results of the survey look on the bank’s dashboard? Depending on your answer to the question 3, the satisfaction indicator will be somewhere in the area of “excellent” or “good!”
The problem of multiple choice questions and how to fix it
There is one common thing about the questions asked – they all are easy to quantify. Actually, any multiple-choice question is already quantified. Let’s call them pre-quantified. It is easy to calculate the percentage of the clients who have chosen certain options, and then use an average of the three answers to get a “Customer Satisfaction Index.” Magic happens and managers have some data to use for the charts for the annual performance report.
Unfortunately, such an approach doesn’t give any clue to what customers liked, and what not, or what their challenges are. Customers don’t think about a company within 3 perspectives. They don’t think separately about “Response time, minutes” or “Customer representative proficiency.” They think about their problem and how easy it was to solve it.
Use open-ended questions to get more insights
With the described approach to the surveys, management will have a slight idea about the problem that the customer faced. A quick fix would be to add an open-ended question that requires a free-form answer.
Here is a real example. The open-ended question of one of our surveys was:
- “What is your key challenge with the Balanced Scorecard?”
And here were the answers:
- “I’m just looking how to get started”
- “I need to understand how it will work for IT”
- “I want to know how to transfer top level strategy to low levels”
- “Getting buy-in”
- “Internal perspective planning”
I think these answers are valuable for R&D and Marketing. For example, we can group them into the categories (users of BSC Designer have Initiatives for this, as described below), count the number of answers in each category, compare answers to the client profiles, and come up with some sort of strategy that will help to better satisfy the client’s needs.
Easy, right? Why don’t companies do this? Two reasons:
- The open-ended questions are known factors of higher abandonment rate (so be careful with them);
- Free-form answers are hard to quantify, as they need to be processed manually.
A checklist for better surveys
Using open-ended questions is a good starting point, but it is important to look at the problem from various sides. Here is a list of 8 ideas that will help you to design better surveys. Feel free to add additional thoughts in the comments.
- Focus on a few well-formulated questions. Twitter with its 140 characters is popular for a reason. Depending on the context of survey, the best number of questions is around 3-5.
- Suggest at least one open-ended question. The answers to such questions are hard to quantify, but at the same time they are the most valuable. Teach your team to read between the lines, and allow customers to go into the detail when it makes sense.
- Be visual. Look at Typeform for some ideas. At least some percent of the participants of your survey will appreciate good visuals and professional design. You will confirm it by observing a lower abandonment rate.
- Select good timing. Asking to fill in a customer satisfaction form when a guest checks out from the hotel and needs to catch a flight is not the best idea. Find the best time to do surveys. For example, in the software business, users are more willing to share their thoughts about the product right after their current problem was solved.
- Let participants learn something new. Don’t just ask questions, engage with the participants of the survey. Avoid a boring design; avoid boring questions; try to teach something, not just ask. In one of the surveys we asked participants to select the KPI practices that they use – later we found out that the survey questions worked like an action plan for many of the potential users.
- Allow participants to ask questions. You need to be creative to get the questions from the clients. You cannot simply add a field: “You can ask your questions now” – it doesn´t work. Try asking an open-ended question that will make customers think in the right direction. We did it by asking clients “What’s your biggest challenge with…?” On the next step, we suggested entering emails to get the answers. Around 23% of the participants shared their challenges and shared their emails to get an answer. That was useful to learn more about their needs, and in most cases we could answer by simply sending a link the relevant best practice article.
- Have some prize at the end. It should not be something expensive. You are not buying a client’s time (it is worth much more). You goal is to give proof that client’s opinions matter. Give some bonus points to their loyalty card, promise a good discount for a front-end product. In one survey, we gave a 75% discount, the front-end product generated almost no profit for us, but the value of new hot leads and the information obtained was high. “Answer the questions to see the results” is also a good motivation to get to the end. Combine ideas 7 and 5 – build a quiz-style survey.
- Track abandonment rate. It is a good indicator of the quality of your survey. People can abandon for many reasons: irrelevant questions, wrong profile of a participant, bad time to ask. Do a/b tests under different conditions to find out what works better in your case. Don’t forget to track the Participation Rate. If you have 20+ questions many will drop the survey without starting.
Employee survey vs. customer survey
For now, we are talking about conducting customer surveys. What about learning what employees think? The same principles can be applied. Normally, with employees we have a higher level of engagement, so we can expect them to be able to spend a little bit more time responding to survey questions. The number of questions can be increased a little bit, and we can focus on some more specific topics.
For example at Google, the manager feedback survey contains 11 agree/disagree questions, and 2 open-ended questions. The survey results are anonymized, which is critical for the success of such initiatives. To be able to discuss other challenges Googlers use additional business tools, such as a “Career conversations worksheet,” a “One Simple Thing” worksheet,” and 1:1 meetings.
Presenting survey data on strategy scorecard with BSC Designer
Conducting a survey is just a starting point of your research. In the best-case scenario, the results of the surveys will be carefully analyzed and presented on the strategy scorecard. Here are some recommendations for the clients of BSC Designer.
Most likely you will use some 3rd party tool to conduct the survey. When you need to get the data to your strategy scorecard, I’d recommend the following approach:
- Open a relevant scorecard or create a new one
- Create a business goal. For example: “Build a better customer service”
- Assign a person responsible for the business goal. This person will receive notifications about upcoming update intervals, and the new initiatives aligned with the goal.
- Create a new indicator – “Customer satisfaction” and align it with the business goal. Assign appropriate an update interval for the indicator (can be monthly or quarterly for example).
- Enter survey results data into the “Customer satisfaction” indicator. It’s normally just one number, so you can do it manually. If there are more indicators you can import via Excel or automate via RESTFull API.
- If you are building an index indicator that is powered by a set of indicators, consider setting ¨weight¨ for the child indicators.
- Use “Initiatives” for the business goal to write down the ideas generated by the open-ended questions. For example, you can sort client’s opinions into several groups, create an initiative for each group, and add the details in the “Comments” field.
In BSC Designer you will also find a data series function. It can help with surveys and will be more suitable for the cases when the performance of certain goal is estimated by several experts using different evaluation criterion (indicators).
A note to the users of BSC Designer
We are now working on a prototype of the survey tool that will help users of BSC Designer to conduct short online surveys, and later update their scorecards with fresh data automatically.
For example, think about this scenario:
- Your business system regularly sends your customers an invitation to participate in the customer satisfaction surveys.
- The answers to the survey are stored in the survey tool and can be automatically transferred to the selected indicator in BSC Designer Online.
If you have some thoughts on the topic, or want to be among first users to beta-test the new survey tool, please follow this link.
Let’s continue the discussion
What survey-related tricks worked for you? Can you recommend a useful tool to conduct a survey, determine survey questions, prize/motivation to finish the survey? Feel free to share your thoughts in the comments.