What do users think about your chatbot? Are they satisfied with the responses it provides? Are they taking full advantage of this tool? Does the conversational agent have a positive impact on recurring contacts? If you don’t know the answers to these questions, it’s probably because you don’t have the necessary key performance indicators in place. These measurements are indispensable for tracking the results of your chatbot, identifying any stumbling blocks and continuously improving its performance. But which metrics should you choose? Here are 16 KPIs to track and analyze in order to determine the effectiveness of your chatbot!
Quantitative key performance indicators allow you to evaluate the effectiveness of your chatbot and the way it’s used by its target audience.
Measuring a chatbot’s Activity Volume means evaluating the number of interactions, from the time a user asks a simple question until a constructive dialogue takes place. This indicator helps answer two key questions.
Usage can be broken down into two types:
The Voluntary Usage Rate is an excellent indicator of your conversational agent’s popularity. It also allows you to verify that your chatbot is well-positioned in the course of the customer experience.
Usage can be broken down into two types:
The Voluntary Usage Rate is an excellent indicator of your conversational agent’s popularity. It also allows you to verify that your chatbot is well-positioned in the course of the customer experience.
The Bounce Rate corresponds to the volume of user sessions that fail to result in the intended “specialized” use of your chatbot. An elevated rate indicates that your bot isn’t being consulted on subjects that are relevant to its area of competence. This should prompt you to update its content, rethink its placement in the customer experience, or both. It’s an indicator that should be observed closely.
The Retention Rate refers to the proportion of users who have consulted your chatbot on repeated occasions over a given period. This indicator can be compared with the typical frequency of client contacts in your particular line of business. It will provide a good indication of your chatbot’s relevance and its level of acceptance among your clients.
This is the number of sessions that are simultaneously active with your chatbot. To get a meaningful measurement, this rate must be weighted with the average number of open sessions during a given period.
This indicator is essential for verifying that you are achieving your goals. If you are targeting a specific population, you can measure the penetration rate for this audience in order to verify that the intended people are making sufficient use of your chatbot. Otherwise, it’s imperative to rethink your change management or customer experience strategies in order to get your users on board!
This is a concrete indicator that will tell you the number of questions your chatbot has answered.
This metric allows you to evaluate the average length of the interactions between your chatbot and its users. The figure will vary significantly from case to case: a chatbot that resolves computer issues or that provides online estimates will require a much longer dialogue than a chatbot that gives the current time in all the cities of the world! If your goal is increased efficiency, this KPI will help you quantify the amount of time saved by your clients, as well as your Help Desk.
At what times of day do users most frequently consult your chatbot? This indicator is particularly helpful, as it often serves to demonstrate how this new 24/7 channel enables you to cover 20, 30 or even 50 percent of the hours during which your user support services were previously unavailable.
The more questions users have to ask, the more time it will take for them to obtain adequate responses. This indicator will help you determine how many questions your chatbot needs to be asked before it can provide the necessary information to its users. Please note that the interpretation of this metric depends heavily on your specific objectives.
If you want to measure user engagement during conversations with your chatbot, you’ll definitely want to observe this indicator. It will allow you to measure the average number of messages exchanged per conversation.
This metric enables you to measure the success rate of a given action performed through your chatbot, for example, clicking on a CTA button or link, filling out a form, proceeding to make a purchase, etc. However, it can only be applied to clearly identified actions for which customized indicators have been created.
This metric measures the number of times your chatbot fails to respond to a question. Such failure may be the result of a lack of content or of your bot’s difficulty in comprehending user inquiries.
What inquiries are most often addressed to your chatbot? Thanks to this statistic, you can adapt your chatbot to specialize in the subjects that come up most commonly and thereby improve its performance. Analyzing recurring questions will help guide your corrective work, allowing you to focus on the topics that are of greatest interest to your users and the mechanisms that will enable you to improve the quality of your bot’s responses, as well as its overall comprehension levels.
To make sense of these quantitative KPIs, you must compare them with other data, particularly the number of calls and the results produced by other channels (e.g. chatbot conversation volume vs. telephone call volume, relative satisfaction rates, etc.). This data will make it possible for you to evaluate the positioning of your chatbot and determine if it’s in the right place with the right knowledge.
Besides quantity, there’s also the matter of quality. The KPIs below will help you measure your chatbot’s “human performance,” including its levels of comprehension, the help it provides to its users and its user satisfaction rates.
Your chatbot will indicate its overall comprehension of user inquiries. This level is constantly evolving, as it depends on:
If your chatbot doesn’t understand an inquiry, it’s either because it has been asked a question that has no meaning for it or because it doesn’t have knowledge in the related field.
For example, a chatbot specializing in computer support won’t understand a legal question!
This rate corresponds to the number of users who were able to obtain the help they needed through the responses given by your chatbot, without subsequently having to call Customer Service. It is calculated based on the percentage of sessions that were successfully completed through an interaction with your bot without being redirected to a live operator. In the process, it enables you to evaluate the level of client satisfaction. This is the equivalent of a call center’s First Call Resolution (FCR) Rate, the percentage of problems that are resolved through a single phone call.
This indicator is very important for analyzing the ROI of your chatbot project.
Finally, it’s indispensable to know what users think about your chatbot. Did it provide sufficient help? Are its users satisfied? There are two different ways you can find out:
This feedback will allow you to calculate two indicators:
These qualitative KPIs should be measured regularly and analyzed over the long term. This will allow you to see how satisfaction rates evolve, whether the recurring questions are always the same, and more.
Knowing these 16 key indicators is essential for evaluating your chatbot’s performance.
However, the best indicators to track aren’t always the same for every company or for every chatbot: it’s up to you to choose the most relevant ones based on your line of business, your goals and the needs of your users. Would you like some guidance to help you figure out which KPIs will be most helpful in measuring the effectiveness of your chatbot?
Guide and inform prospects with a chatbot on your web portals. Our intelligent conversational software will interact with web visitors to guide them to relevant information.
Discover
