FAQs

The main goal of this document is to clarify frequently asked questions from our clients.

What is the role of eva nlp in eva?

eva nlp is the cognitive engine of eva, responsible for training and prediction of intents and entities.

What is the difference between intents and entities?

An Intent is a categorization of a user intention and is generally represented by an action the user wants to take, regarding some information.

Example: Add example of an intent usage

An Entity is a group of specific information used to describe or specify an action. Entities are usually used to desambiguate the information the user is providing to the chatbot. There are thre types of Entities provided by eva nlo:

• Synonyms

• Pattern

• System

Example: Add example of an entity usage

When should I use intents? And entities?

The recommendation is to use intents when there is a goal or aim to the user, when typing their messages or questions.

Whenever this action expected by the user has different variations or specific information that needs to be provided to fully understand what he wants, you should use entities.

What types of entities are available in eva nlp? In what languages?

Synonyms Entities

blabla

Like the name implies, this a text matching entity, when you to capture especific words or terms. Example:

Pattern Entities

This type of entity is usually used when there is a specific pattern of information you want to capture from the user's message Example

System Entities

Pre-built entities offered by eva nlp, that can be used to capture informations like dates, addresses, numeric information, etc... A list of all System Entities can be found (link to the public documentation)

How does eva nlp calculates the confidence scores for intents?

In intent classification tasks, a confidence score represents the likelihood that a given sentence/utterance sent by the user is the correct Intent. This score varies from 0% to 100% and it is distributed across all possible predictions(intents).

Add an image here, explaining the distribution of confidence score across predicitons

How should I tune the confidence score threshold?

The confidence score from eva nlp predictions can vary, depending of various factors:

• Unbalanced dataset

• Volume of utterances per intent

• Quality of intents

It is important to note that this score is directly related to the quality of the dataset created. For information on best practices go to (link to document for best practices)

You just said that the confidence score varies across all intents, why is that my intent has a 100% confidence score?

In eva nlp, all created utterances are stored and in a dictionary-like object, which is used to return the intent directly in cases where the user types the exact same utterance. This feature is called exact match.

In that case, since there is not prediction from the intent classification model, we assume that if the user sends a message that is exactly the same as trained, it has 100% confidence that that is the correct prediction.

What is the pre-processing step in eva nlp?

In eva nlp, there is a pre-processing step that consists of:

• Removing bad characters

• Analyzing bad pattern entities

What about stop words? How does they affect the predictions?

Stop words are words or terms that does not add context or more information in a sentence. Example of stop words are:

• pronouns

• articles

In classic intent classification models, each word needs to be mapped to a specific representation, which led to various problems with typos and the frequency in which stop words appears. Our solution works with a different approach, which considers not only the words but the sequence in which they appear. In other means, we consider the context when generating those representations and training our model

Because of that, stop words are a important factor in maintaining that context thus, we keep them in our solution.

What style of sentences (utterances) should I use?

Like explained above, context is an important factor in creating a good intent classification model in eva nlp so, is a good practice to create sentences that have context in them (meaning that a group of utterances from an intent should be of the same context).

By that logic, It is also recommended to create small/medium size sentences in the training set, instead of utterances with one or two words only.

What is recommended to do when I have intents that shares similarities?

Depends on the case. The rule of thumb in those cases is to use entities to disambiguate between similar sentences, when possible.

In general, it is recommended before creating the intents dataset, to understand what is the type of information you are expecting the user to ask for. For example:

• Intent: Ask for information In here you add examples on how a person usually asks for information

• Entities: What type of information is the user talking about

  • Traffic

  • Office Hours

  • Education

If it is inevitable to create similar intents, it is important to validate (using eva automated tests) the accuracy of the bot (specially for those intents) and the mean confidence score, adjusting its threshold accordingly.

What are the languages supported by eva nlp?

You can check all the supported languages here: (link para a documentação com as listas de linguagens)

How can I measure the performance of my chatbot?

Does eva nlp controls the flow of conversation in eva?

No, the conversational flows are controlled by the user that is creating them in the cockpit

How do I deal with the rate of false positives in my chatbot?