Obtaining your Chatbot becomes a werewolf? Then what is your Silver Bullet?

Conversational artificial intelligence and machine learning is more than a fantasy now. Among several cutting-edge practical AI offerings, conversational AI technology is evolving at a rapid pace to support modern enterprise demands for intelligently managing consumer and employee communications. In the world of conversational AI, chatbots are more preferred than ever before, but the fact remains that acquiring cognitive chatbots is still cumbersome, due to several factors like limited AI- and machine learning-ready data, challenges in natural language understanding and vendor qualification, complexity in the platform selection process, limited engineering and development support, bottlenecks with the traditional operational model, and enforced ethics, amongst others.

To become successful, choosing a proper platform is pivotal for ensuring intelligent, scalable, and cost-effective chatbot technology. Decision-makers are looking for an easy buying process with hassle-free configuration, training, and deployment to harvest ROI as quickly as possible with minimal effort and expense, so they are compelled to explore multiple options. The common choices are one of two options, buying or building the chatbot.

To understand the real difference between the buy and build approaches, we need to start from:

1. What is a chatbot?

2. What is the confidence and accuracy of a chatbot?

A chatbot is an independent computer software program based on natural language understanding that is designed to simulate human-like conversations with users via text, voice, or image inputs. Machine learning (ML) and Natural language processing (NLP) are the most commonly used AI subsets for "AI-powered" chatbots, but in contrast, "rule-based" chatbots are also playing a significant role in the market.

Under conversational AI, a chatbot with preconfigured actions and pre-written keywords is used to understand user queries to perform expected actions. These are called "rule-based" or "decision tree" chatbots. Such a chatbot platform has limited dialog flows but still delivers predictable results within its limits.

Rule-based chatbots can be developed and trained rapidly to work with limited enterprise data and can also be integrated seamlessly with enterprise systems. Their dialog flows can also be configured, streamlined, and managed with less cost and effort. There is a similar chatbot called "retrieval-based" chatbot in the market. Instead of Decision-Tree, it uses a repository of predefined responses and a ranking model to pick an appropriate response for the user input.

With the help of machine learning and NLP, “AI-Powered” chatbots can read, understand, and process user queries like a human without any pre-loaded instructions. It performs skills such as "utterances", "intents", "entities", and "variations".

Utterance: The entire statement from the user

Intent: The user's intention (verb in the statement)

Entity: It modifies an intent (noun in the statement)

Variations: The different ways of framing the statement

AI-powered chatbots will consume a long time to build and train but can bring massive savings in the long run. Such chatbots can deliver a rich user experience by establishing human-like conversations. They can learn from experience and continuously improve over sample data training. Like a human, they can make some self-decisions through supervised, semi-supervised, or unsupervised learnings. Most importantly, they can detect false positives and false negatives in their conversations to detect potential misbehavior.

Now let's see a real-time example for a chatbot's confidence and accuracy. A user, who is a sports enthusiast, bought a personal chatbot to catch up on the sports trends and match forecasts. This chatbot was trained with extensive sample data and configured for a unique personality which is opted by the user. Also, this chatbot is connected with several real-time data sources like analyst reports, user blogs, and sports news websites to generate custom predictions.

In general, a chatbot's accuracy could be as high as 99%, but anything less than 40% is low confidence. In the above example, both accuracy and confidence levels are high. For the same example, the chatbot would have responded like the  below illustration, if it has a low confidence score but a high accuracy rate.

Likewise, in case of high confidence score and low accuracy rate:

The confidence score is a threshold indicator, where the threshold determines the highest and lowest matching scores acceptable to trigger the respective actions. If the matching score falls below the threshold, then the chatbot will either provide a low confidence response or it will trigger a fallback interaction that either asks the user to repeat the query or transfers the query to human response teams for support. In case of a high confidence score, the chatbot will trigger the actions to produce the desired results.

The confidence threshold can be dragged up and down based on the availability of AI-ready sample data. Importantly, it is not necessary to have a high threshold in all scenarios. A low confidence score value is recommended only in a very limited number of use case scenarios. An inappropriate confidence threshold will bring a negative impact and leads to chatbot misbehavior.

Chatbot accuracy can be misinterpreted easily. Even though accuracy is a key driver, this alone will not create any impact. Accuracy of a chatbot can be defined as the % of utterances that had the correct "intent" returned for the identified actions. There are several cross-validation methods to test and validate chatbot accuracy, some of which are K-Fold, Monte Carlo, and 80–20% split, among others. A higher accuracy rate will bring better end results which increases customer satisfaction.

Let’s gear up with basic questions

1. How to identify chatbot necessity?

Defining long-term AI vision strategy with an adequate budget is the steppingstone to this journey. The objection of this journey varies from enterprise to enterprise. Among others, it can be one or more of the following:

• Improving or delivering a consistent user experience
• Increasing operational efficiency
• Reducing operational cost
• Reducing time to market

2. How to set a vibrant vision and strategy?

ACID test the visibility of use cases and connect with application owners, business leaders, IT heads, and other required stakeholders at an early stage to set a solid base for success. This will ensure that stakeholders are comfortable with the deployment of a chatbot in their enterprise.

3. How flexible is enterprise architecture?

For each chatbot use case determine the enterprise systems that will be impacted and assess their suitability to efficiently connect with the chatbot.

Primarily IT, network, data security, and corporate headquarters must collaborate for fruitful results. In terms of information security, data types should be classified as:

• Transaction data: Information that is enterprise-specific and confidential.

          For example: Customer account number, SSN, etc.

• Conversational data: Information that is generic and does not pertain to the enterprise.

          For example: Weather

4. How important is the Communication Model?

The enterprise needs to consider the end user’s aptitude and appetite while selecting the chatbot channel. When the chatbot is deployed in an enterprise, persuasive marketing tactics should be conducted to win end user’s trust.

5. What is the most important factor while procuring a chatbot for my enterprise?

The decision to purchase the raw platform technology or going with a productized platform offering for a chatbot is really a critical process for almost all the technology leaders/decision-makers in the enterprise.

Let's find the "Silver Bullet"!

Decision-makers who are ready to take the risk with the combination of an in-house AI engineering team, a well-defined strategy/roadmap, and successful track records in AI deployments in the same enterprise, may get the subscription directly from platform providers to build their chatbots. Some of the top providers are:

• Google - "DialogFlow"
• IBM - "Watson Conversation Services"
• Amazon - "Lex"
• Microsoft - "LUIS"

There are other types of platform providers in the market who offer application-specific platforms. It should be noted that such platforms may go well only with their native applications. In order to avoid the complexities of scaling up chatbots as an enterprise-wide offering or for cross-domain culture, the following factors can be considered by decision-makers adhering to a build approach. A few examples include:

• Salesforce - "Einstein"
• SAP - "SAP Conversational AI"
• ServiceNow - "Conversational bot platform"

Typically, in cases that involve platforms such as Einstein and SAP Conversational AI, the chatbot build process is a time-consuming activity to achieve success. In order to simplify the process, the decision-makers with build strategy can consider the following factors:

•  “Data” is first – Data is the key to an effective AI implementation. Without healthy and meaningful AI-ready data for chatbot training, the chatbot performance will be sub-par.
• “Integrations” are critical – Back end systems and databases should be refactored securely and effectively for chatbot integration, smooth data movements, and automation.
• “Channel” makes the difference – Building and publishing an effective chatbot front end will boost its productivity.
• “Algorithm” is the core – A powerful NLP platform is the base, but the algorithms and models for dialog flows need to be created and trained for their intended actions.
• “Privacy” is everywhere – Information security will protect the business and it is critical to take full control of        how the data is/will be used and who can access it? Chatbot with poor data store or transfer policy will bring huge business loss and non- compliance/regulatory issues.
• “Maintenance” for long life – Extensive planning and investment is required to manage and maintain chatbot. Like Health monitor, performance monitor, addition, deletion or modification of use case, chatbot training, algorithm management for logical controls, etc.
• Start Small to “Scale” – Target low hanging fruits to test user acceptance and chatbot performance.
• “Human” must collaborate – For effective results and rich user experience, the chatbot should be designed to dispatch requests to human agents for all unknown /not trained asks.
• “Training” brings power – Chatbot efficiency will increase over continuous data training. Investment for training automation is highly recommended.
• “Personality” is vital – Effective chatbot with poor personality is like hiring a skilled employee with bad work ethics, which will defeat the purpose.

Apart from a chatbot’s confidence and accuracy, there are other factors to consider while building a chatbot:

• Performance load
• Security
• Precision
• Response speed
• Error management
• User feedback for chatbot optimization

Last but not least, the decision-makers with a buy strategy can play a different game which is more facile and foolproof. There are some platform providers in the market that offer their productized chatbot solution as a holistic product with professional services. These companies took most of the complex tasks to come up with a viable chatbot platform for the buyers.

5 KPIs to measure:

1. Out of the Box Use Cases: Like the way, in which an employee's productivity is dependent on their knowledge and learning, the productivity of the chatbot is dependent on error-free configuration, which comes from ready-to-use and pre-trained and tested use cases. A user-friendly portal with adaptors will support hassle-free new use cases configurations as well as training.

2. Natural Language Understanding: How effectively can the chatbot understand user queries? Basically, this skill comes from the underlying NLP engine for "intents", "entities", "utterances", and "variations". A famous feature is the FAQ parser, where the chatbot is simply connected with any knowledge source system to answer the user's questions.

3. Deployment: How effectively can the chatbot be deployed in the enterprise? Ideal time to market (TTM) with meaningful milestones can thrive well. Apart from effective dialog flow configuration with optimal performance load, precision, response speed, error management, and user feedback collection, the deployment phase contains four primary tasks: Integration, testing, information security, and channel.

a. Chatbot Integration: Integration with external and in-house enterprise applications should be done using off-the-shelf integration adaptors. This skill comes from the number of robust tools and adaptors for the end to end chatbot integrations.

b. Testing: Extensive dialog flow testing with the right sample data and user acceptance testing in the production environment with a valid "service ID" for the chatbot. The service ID should have read/write access like a real employee ID.

c. Security: In general, security and data protection itself is a vast area. In terms of AI, where machines take control of human tasks to serve the end user, security policies must be validated completely in all segments such as integration, information transfer, data storage, and data extraction, among others. Extra care should be taken for cloud deployments, integrating chatbots from cloud to on-premise systems should be achieved after DMZ assessment to bring the right middle layer solutions, like middleware servers, VPN tunnels, and secure gateways, among others.

d. Chatbot Channels: Nowadays, advanced chatbots support multiple channels. The enterprise needs to consider the end user's comfort zone for selecting the channel. Some popular channels are:

• Desktop applications (Example: web browser and email, among others)
• A dedicated mobile and desktop app
• Voice device (Example: Alexa, Google Home, and Siri, among others)
• IVR (Example: telephone and voice, among others)
• Chat Engine (Example: MS Team and Jabber, among others)
• Social Media (Example: Facebook messenger and WhatsApp, among others)

4. Pilot: A pilot run for a short time period will help the enterprise to test and measure the real-time performance of the chatbot in a production environment.

5. Pricing: The most common pricing models in the chatbot market-

• Monthly license fee: Monthly cost based on limited or unlimited usage
• Pay-per-use: Cost based on usage and number of application integration
• Pay-per-performance: Cost based on successful conversations

Finally, what is the big fish?

Obtaining an enterprise-wide successful chatbot is a rigorous task for most AI experts and decision-makers. So, the leaders who are not in a rush and are prepared for all unexpected surprises can build their own chatbot. The aim is to lay out a clear vision plan and stick to that with patience. Leaders with a buy strategy will experience low or no risk in this journey, but they need to evaluate the KPIs to qualify the right provider for their chatbot needs. Either with the build or buy strategy, leaders should focus on the factors discussed above before making the decision.