How Intuit is using Dashbot Analytics to Improve Customer Service Chatbots. — Dashbot

Natural Language is the Future

The Quickbooks Assistant chatbot is designed to help users get the most out of Quickbooks in three ways:

  • Help users understand what features are available, how to use them, and how to troubleshoot issues, if needed.
  • Help users understand their financial standing by providing instant access to any question they have about their money or taxes.
  • Answer general tax and business questions for which users often search.

The chatbot is available in the native iOS and Android versions of Quickbooks, as well as Quickbooks Online on the web.

Conversational interfaces require specialized analytics

Intuit saw the value in Dashbot’s conversational specific analytics. With Dashbot, users can traverse conversational paths, better understand users Intents, and optimize Natural Language Processing (NLP) response effectiveness.

One of the features Intuit makes heavy use of is Dashbot’s Message funnels — via the top messages in and out from a chatbot. Through navigating the flow of users messages and the bots responses, Intuit can better understand users interests and Intents, as well as see how well the chatbot is responding and make improvements.

Improving NLP Response Effectiveness

Intuit looks closely at their fallback percentage, the percentage of user inputs that don’t match an Intent in their NLP engine. A lower fallback percentage would mean their NLP is matching at a higher rate — handling more of the users’ messages. However, it’s not just whether there is a match; it needs to be a correct match. It’s important to look at mishandled Intents in addition to unhandled.

To improve the process, Intuit makes use of Dashbot’s Phrase Clustering and Unhandled Intent reports.

With Phrase Clustering, Dashbot groups similar messages together based on proprietary data science techniques. The Phrase Clusters can help identify unhandled and mishandled Intents. For example, a cluster for “help” may include a hundred different ways users ask for help — some of which might map correctly to the help Intent, some which are mapped incorrectly to a different Intent, and then others not mapped at all — the unhandled messages. Intuit can quickly view these clusters and update their NLP engine to support the various ways users ask for help.

“Dashbot phrase clusters are by far the best. We tried a competitor who had a similar tool that was useless — they were not accurate at all. We’re really impressed with the accuracy of Dashbot. Even our own internal team’s clusters weren’t as tight or accurate,” said Spencer Uttley, Senior Product Manager, Intuit Quickbooks.

Phrase Clustering saves a tremendous amount of time and is much more accurate. Previously, Intuit had to manually pour through spreadsheets of messages, trying to identify all the similar messages on their own. This was a time-consuming, laborious task that could result in overlooked messages as well. The goal is to prioritize Intents and identify which improvements can have the biggest impact on reducing the fallback percentage.

“Dashbot enables us to accelerate our NLP optimization through Machine Learning,” said Uttley.

Handling Fallbacks Gracefully

Intuit is able to track the success of these article suggestions using Dashbot’s event tracking. Event tracking enables Intuit to capture and report on which external articles were clicked on from the bot, as well as the user message that lead to the suggestions being displayed.

In addition, Intuit prompts the user for feedback on the suggestions. All of this information can be used to improved the response effectiveness.

User Satisfaction is Important

Intuit can quickly see the success of the responses through the Dashbot Message Funnels. One can traverse the flow from the chatbot response to the thumbs up, down, exit, or other response to better understand the users’ satisfaction.

In addition, Dashbot’s sentiment analysis tools help identify users’ overall feelings toward the chatbot as well as individual transcripts based on sentiment. The Intuit team often looks at the positive sentiment transcripts for inspiration on how the chatbot has been able to help users.

Dashbot is helping enable Intuit to improve the chatbot user experience through automated NLP optimization to reduce the fallback percentage, and reduce the overall manual labor involved. This in turn is enabling Intuit to provide higher quality answers and an overall better user experience for their customers.

About Intuit

About Dashbot

Chatbot, Voice Assistant, and AI Entrepreneur; Former CEO/Co-founder Dashbot

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store