Introduction
...
Required sensors: Mic audio
Required actuators: None
Required service(s): stream_audio
or audio file
audio length + audio type (bytestream)
...
The following drivers need to be
...
running
...
if testing locally;
computer-robot,computer-speaker,andcomputer-microphone.
Output
The output solely depends on your project and the set-up of your intents and entities of the Dialogflow agent.
confidence level - 0-100
transcript - string
entities + their type
intent - string
Parameters
Dialogflow Keyfile path - string
Dialogflow Project ID - string
Service Configuration
...
Our service communicates with a Dialogflow agent to achieve its intended purpose, and it does so by using a project ID and a key file. However, however, if you happen to have them, you may skip the this section.
The following steps will help you get the required items:
Create a Dialogflow agent by clicking the following link: https://dialogflow.cloud.google.com
Use the ‘Create Agent' button at the left top to start your first project. Press the settings icon next to your agent's name at the left top to see the Project ID.
Follow the steps here to retrieve your private key file in JSON format.
The following parameters can be modified to configure the service:
In order to run this service, the following parameters are needed:
dialogflow_agent_id
dialogflow_key_file
...
Initialisation
...
Initialisation
Setting up intents
The main items of interest are the Intents and the Entities. Intent is something you want to recognise from an end-user; here we will show you an example of an intent that is aimed at recognising someone’s name.
...
abstract away + concrete example
When creating an intent, you can name it anything you like; we go with 'answer_name' here. Below 'Action and parameters', you should give the name of the intent that will actually be used in your program. Here, we also make that 'answer_name'. Moreover, it is useful to set a context for the intent. A context is set by the requester in order to indicate that we only want to recognise this specific intent and not another one. Usually, in a social robotics application, the kind of answer we want to get is known. We match the name of the (input)context with the name of the intent and thus make it 'answer_name' as well. By default, Dialogflow makes the context 'stick' for 5 answers; we can fix this by changing the 5 (at the output context) to a 0. Now we arrive at the most important aspect of the intent: the training phrases. Here you can give the kinds of input strings you would expect; from these Dialogflow learns the model it will eventually use. You can make a part of the phrase into a parameter by double-clicking on the relevant word and selecting the appropriate entity from the list. It will then automatically appear below ‘Action and parameters' as well; the ‘parameter name’ there will be passed in the result (we use 'name’ here). The system has many built-in entities (like 'given name'), but you can define your own entities as well (by importing CSV files). Our complete intent example thus looks like this (note: using sys.given-name is usually preferred):
...
Using the service
In order to use our service for your purposes, there are two classes with which you must interact, namely BasicSICConnector and ActionRunner. You can find the details of these classes here. You may also need a class to manage speech_recognition attempts and a callback function for retrieving a recognized entity from the detection result.
...
You have the correct agent name and keyfile(path) as parameters for an instance of the class Example in the example file and are passing it as parameters when creating an instance of BasicSICConnector.
You have the Dialogflow service running, via the command “docker-compose up dialogflow”, services running and you have the relevant local devices running.
Example
We have provided a The following file, https://bitbucket.org/socialroboticshub/connectors/src/master/python/speech_recognition_example.py, is available for the purpose of demonstration. There are two classes worth paying attention to.
Recognition Manager:
Recognition manager manages speech recognition attempts and is used by the class Example
Example:
Two types of questions are included. The first is dealt with in this example, the first being an entity question where we are interested in a specific entity in the answer. In this case the point of interest is the name of the person that is interacting with the robot. The second is a yes or no question. The answer can be user, and the second being a yes, no, or don't know (or any synonyms).This class interacts with two external classes, BasicSICConnector and ActionRunner. An instance of BasicSICConnector is needed to interact with the Social Interaction Cloud, and ActionRunner to execute the desired actions appropriately. In order to run this example, don’t know question.
Creating an intent
In order to deal with the first question, an intent needs to be set up. Intent is something you want to recognise from an end-user, in our example that would be the name of the person. The following steps will set an intent of your Dialogflow agent:
Begin by naming your intent, you can name it whatever you wish; we have named it ‘answer_name’.
In the section Action and parameters, you should name the intent to what you will use in the program. In the given example, we have used the name ‘answer_name’.
In the section context, this section indicates that one is looking for a particular intent. Match the (input)context with the name of the intent, ‘answer-name’ has been assigned in this particular example. Thereafter, set the output(context) to 0, this option is set at 5 by default.
In the section Training Phrases, there are empty input strings, which when filled with expressions you would expect one to express, Dialogflow will learn from these to make a model which would also include phrases similar to the list of expressions you gave. Our example has used a list of phrases that you can find in figure 1.
Select a word from an expression as a parameter by double-clicking on it and selecting the appropriate entity from the list. It will then automatically appear below ‘Action and parameters' as well; the ‘parameter name’ there will be passed in the result (we use 'name’ here).
Our complete intent example thus looks like this (note: using sys.given-name is usually preferred):
...
With these steps, the intent of your Dialogflow agent would be set up successfully for the example file. All that remains now is to ensure to pass your agent name and keyfile(path) as parameters when creating an instance of class Example.
For each intent that you create, make sure that: abstract away + concrete example
Events
Events that the There are no events that this service creates and can have listeners.
Known Issues
There is a rare bug where sometimes Dialogflow will suddenly only respond with ‘UNAUTHENTICATED’ errors. Restarting Docker and/or your entire machine seems to be the only way to resolve this.
...