What is machine learning? Natural language processing and understanding explained.

Summary: Machine learning enables artificial intelligence to learn for itself instead of needing to be taught. Natural language processing and understanding aim for the bot to derive literal and implied meaning from conversations, allowing for more complex use.

Machine Learning is used for a vast range of things, but today we’re talking about its applications and uses in live chat. It’s a form of artificial intelligence (AI) that enables systems to learn by themselves and continuously improve without being explicitly programmed.

The process of machine learning involves a lot of data. Everything from examples, experience and direct instruction, to allow the AI to get smarter. When used in live chat, chat transcripts enable the bot to improve, meaning that the more chats it takes, the better it becomes.

The primary aim is to allow the chatbots to learn by themselves, without needing constant human assistance. By doing this, they will be able to adjust their decisions accordingly and choose the right responses.

Natural Language

There are two main types of machine learning – Natural Language Processing (NLP) and Natural Language Understanding (NLU).

Human language is complex. Words often have duplicate meanings or are shortened, human error means there are often spelling and grammar mistakes, abstract concepts, implications and even sarcasm. A simple rule-based chatbot cannot begin to recognise these things, and can only respond with set phrases to keywords. That’s where Natural Language comes in.

Natural Language Processing chatbots find a way to convert the customer’s speech into structured data which can then be used to find the most appropriate answer.

Natural language understanding

The bot is programmed using entity, intent and context.

Entity – an idea or concept is given to the bot, which is the entity. This could be ‘pizza’.

Intent – the intent is what the use is implying, and what action the bot should perform. E.g. if the user asks ‘do you do pepperoni pizza?’ the user is wanting to order a pizza.

Context – the NLU won’t remember the question it has asked once it is sent, so a state of phrases is stored to ensure the bot will retain the concept of the conversation.

Natural language processing

Natural language processing takes things to a new level and enables the chatbot to continuously get smarter.

Tokenisation – sentences are split and words are separated into different pieces (tokens) which are useful for the bot.

Sentiment analysis – here the bot is trained to recognise moods and emotions. It can learn whether the customer is having a positive or negative experience and whether they need to be transferred to a human.

Normalisation – the bot processes the text to check for common spelling mistakes, errors and typos.

Named entity recognition – the chatbot looks for ‘entities’ within the sentences that the user has types, such as product names or customer information.

Dependency parsing and speech tagging – the bot looks for subjects and objects such as verbs, nouns and common phrases to find related phrases they may be trying to convey.

Final thoughts

Although machine learning is still very much in its infancy, the technology has great potential and is improving every day. For simple chat applications, a rule based chatbot can be a perfect solution, but to level things up and handle more complex queries, using artificial intelligence and natural language processing is the way forward.

Interested in seeing how a chatbot can work for you but not sure where to start? Chat with us.