Virtual Assistants are taking the digital transformation journeys of the enterprises by a storm. It’s no understatement that, COVID-19 has been the disruptor-in-chief, making the importance of enterprises picking the right partners in their digital journeys is all the more crucial. What’s more important in a partners’ capabilities than NLP, which forms the core of the very definition of a Virtual Assistant?
First things first, NLP implies Natural Language Processing. NLP is essential to understand the language written naturally by humans and process it for further actions. Imagine a customer typing on an airline Virtual Assistant
- “I want to book a flight from Mumbai to Delhi”
NLP would be used to decipher Mumbai and Delhi as Origin and Destination respectively and move ahead to fulfil the customer’s intent of booking a flight ticket.
Haptik is proud to announce the most recent recognition for it’s NLP capabilities, coming from the academic fraternity. EMNLP, the most prominent NLP conference, has accepted Haptik’s research submission and recognised the stellar performance of Haptik’s NLP.
Haptik compared its performance with multiple other platforms like RASA, LUIS, Dialog Flow on a large set of real-world customer queries. Below is the snapshot of the performance -
The results indicate that Haptik is at the top of the pack, and even beats some competing platforms by a significant margin when it comes to the accuracy of the NLP on real customer queries. Since this performance comparison is done on real-world customer queries, the likelihood of such performance translating to real-world Customer Satisfaction (CSAT) is very high.
Additionally, below is the snapshot of the comparison done for scenarios after the training data of the virtual assistant was reduced by a significant margin.
Scenarios containing lesser training data regularly emerge for virtual assistants in their initial few months yet it is a very crucial phase. As virtual assistants interact with more and more live customer queries, their training data increases and virtual assistants mature.
Moreover, scenarios containing lesser training data also occur when enterprises are trying to automate some uncommon use-cases, and training data for those use-cases may not be readily available. In such scenarios, it’s important to ensure that the customers who interact with the Virtual Assistants in the “Learning Phase” continue to achieve high Satisfaction Rates (CSAT).
Through this research submission, not only did Haptik prove its performance but also filled a crucial gap in the NLP Community - How to determine whether an NLP A is better than NLP B? To answer that, Haptik released HINT3 - a large set of real-world customer queries (with the utmost regard for customer privacy). Enterprises and Vendors can use it to benchmark and improve the performance of their NLP Technology.
Hitherto, the comparisons were done on artificial customer queries, created specifically for the purpose of comparison. HINT3 helps the industry escape this predicament. With EMNLP’s recognition, HINT3 has the potential to emerge as a major benchmark for NLP performance globally, as seen in RASA’s analysis. Both the comparisons shown above were duly accepted at EMNLP 2020 and are fully reproducible with the steps mentioned.
However, it’s natural to be curious about how Haptik achieves such high performance when it comes to accuracy. Well, as it goes for most questions, even this question doesn’t have one single answer.
a) Firstly, as opposed to most vendors that build Virtual Assistants, Haptik’s NLP model uses a triplet of models to understand and answer a customer’s query. This enables the IVA to simultaneously achieve a word level and a contextual understanding of the user’s query. These triplets of models are also trained out of the box on industry-specific verbiage, thus providing a sound understanding of industry-specific contexts.
b) Secondly, Haptik uses state of the art “Training Data” augmentation techniques, to algorithmically expand the human-provided training data. This helps Haptik achieve superior performance, even when training data is in low quantity.
c) Thirdly, Haptik also trains its IVAs on the negative customer queries (in NLP parlance, adversarial queries). E.g For a user utterance Please book a flight for me, a negative user utterance would be Please sell a flight for me or Please book a train for me.
Long Story Short - With Haptik, you can be rest assured that the NLP would leave no stone unturned in ensuring top-notch customer experience, be it in the early days of a Virtual Assistant going live or in its steady state. The EMNLP accolade is a continuation in a series of accolades Haptik has received starting with the RAISE Award from the Government of India, and an industry-leading performance here.
Thus, the above innovations in the NLP capabilities of Haptik, coupled with a superbly talented NLP team, enables Haptik to not only position itself as one of the leading Virtual Assistant platforms, but also constantly push for raising the bar in the Virtual Assistant industry, at large.
This article has been written by Shrey Jain, Senior Product Manager at Haptik.
Interested to know more?
