Not very long ago, Google’s Meena chatbot caught the world’s attention through its human-like conversational behavior and its very close sensibleness and specificity average (SSA) index to humans (the SSA for Meena was 79% in comparison to the 86% of humans).
The researches proclaimed Meena chatbot to be the most sensible AI-based chatbots in the world, capable of indulging into human-like dialogue exchanges.
However, it looks like Facebook’s AI chatbot, Blender, has broken that record.
Developed by Facebook Artificial Intelligence Research (FAIR), the team informed on April 29 that it has built an open-source chatbot called Blender, the best AI-chatbot in the world.
They claim that Blender beats Google’s Meena as the best chatbot in the world.
A Lot Of Investments Are Being Made In The Field Of Virtual Assistants And Chatbots
Despite the progress that is taking place in the field of chatbots and virtual assistants, the most popular ones are terrible conversationalists. Chatbots, developed till now, are mostly task-oriented and very boring. Chatbots, for sure, are be able to service all your commands, but they lack the fine touch of human companionship in them. As these bots are becoming increasingly popular in widespread industrial domains, researchers are focusing on improving their inadequacies and power them with human-like quotient to make them conversational experts.
Much like Google, the leading tech company, Facebook has been heavily investing its money and resources in improving its Natural Language Processing (NLP) technologies for a few years now.
Facebook now claims that its efforts have been paved off by their new AI-based chatbot, Blender.
What makes Facebook’s AI chatbot, Blender, a top chatbot contender
Blender is an open-domain chatbot named after its ability to merge multiple conversational skills at once. The chatbot is still a work in progress under FAIR (Facebook AI Research) fed with a neural model of 9.4 billion parameters, which is 3.6 times more than Google’s latest chatbot, Meena, so big that it cannot fit on a single device and must run across two computing chips.
The AI team at Facebook used what they called Blended Skill Talk (BST) to merge various chatbot abilities. It has been trained using public domain conversations consisting of 1.5 billion Reddit human exchanges. This powers the foundation of generating responses in dialogue. As per Facebook, the neural model will become larger in the years to come. Their goal is to develop a chatbot with a stable personality that can converse naturally, understand the emotional context of the user, and match with it. It should be powered by human emotions that don’t seem to be offensive or inappropriate to any user.
As explained by the researchers, “Blending all the human skills are being a difficult challenge because systems must be able to switch between different tasks when appropriate, like adjusting tone if a person changes from joking to serious.” They continued saying, “Our new BST data set provides a way to build systems that blend and exhibit these behaviors. We found that fine-tuning the model with BST has a dramatic effect on human evaluations of the bot’s conversational ability.”
How Blender Occupies An Upper-Hand On Google’s Meena Chatbot
Facebook’s Blender has completely thrown Google’s Meena chatbot out of the water. Launched in January 2020, Google Meena chatbot was said to be a powerful chatbot, probably the best in the world. However, in four months, Facebook proved its might through their AI-chatbot, Blender. Now, Facebook’s evaluations prove that,
- 75% of evaluators found that Blender is more attractive than Meena
- 67% of evaluators believe that Blender performs more like humans
- 49% of evaluators cannot distinguish between chatbot Blender and real humans initially
What distinguishes Blender from Meena is that the former can assume a persona, show empathy, and discuss any topic with the user. This is one of the biggest popularity aspects of Blender.
Talking about Blender’s impressive results, this Facebook chatbot is packed with the huge scale of training data (1.5 billion public conversations), an additional dataset for blending skills (by incorporating the Blended Skill Talk), and amazing strategies.
Blended Skill Talks (BST) is a very fine model that enhances Blender’s dominance over any other chatbot. It consists of several strategies like beam search, next token sampling, n-gram blocking, and carefully selected the search hyperparameters.
It is constructed using the following skills:
- Understands the personality of the user (PersonaChat)
- Ample of knowledge and general learning (Wizard of Wikipedia)
- Can display empathy (Empathetic Dialogues)
- Ability to blend all the above-mentioned skills (BST)
Like Any Other Chatbot, Blender Is Still Learning And Prone To Make Mistakes
As agreed by the Blender researchers, the chatbot is not ready for public display, despite its excellent conversational ability and human-intelligence skills. The team has evaluated the chatbot’s performance for a 14-turn conversation only. Like all the other chatbots, it would make the conversation repetitive and dull over the course of several days and weeks.
It also suffers from several other problems. For example, Blender forgets earlier conversations. Furthermore, it uses the standard transformer architectures which have a saturation limit of 128 BPE tokens of chat history. Therefore, it cannot expand on its learning pool about a user’s persona and their previous conversation.
The issues clouding this chatbot doesn’t end here.
It also has the ability to collate facts, which is also a limitation in deep learning. Just like any other chatbot powered by deep learning, Blender does hallucinate thoughts or make up facts. Perhaps, it generates sentences based on statistical correlation rather than conversational knowledge.
Blender And The Future Of Chatbots
Developing Blender is going to take a series of more data, to make a conversation pleasing and admirable, and prevent any disaster like the same committed by Microsoft’s chatbot, Tay.
The researchers believe that they are far from achieving human-level intelligence in a dialog system. Even the best models and most data fed chatbots can make mistakes.
The team hopes to experiment with better safety mechanisms like a toxic-language classifier, that can double-check the chatbot’s response to a user message. They are also interested in developing more sophisticated conversational agents that can even respond to visual cues. The project is called an Image Chat that can converse with a person and understand their personality using photos.
The Task Of Developing A Human-Like Chatbot Is Arduous
Human conversations are not limited to a particular topic. They have infinite possibilities, which makes the task of developing a chatbot that can mimic human dialogues, a formidable effort. However, with the advancement in technology like NLP and deep learning, chatbot service is also said to improve.
Empathetic and knowledgeable chatbots are to be developed in the near future.
As for Facebook’s chatbot, Blender is going to undergo several improvements before being publicly released.
For Google’s chatbot, Meena, a new challenge awaits from their potent competitor, Facebook.