Page Comparison

Introduction

The dialogflow service allows using the Google Dialogflow platform within your application. Dialogflow is used to translate human speech into intents (intent recognition). In other words, not only does it (try to) convert an audio stream into readable text, it also processes this text into an intent (possibly with some additional parameters). For example, an audio stream can translate to the string "I am 15 years old", which is, in turn, converted to the intent 'answer_age' with the parameter 'age = 15'.

Docker name: dialogflow

Input

• Required sensors:

...

• Microphone

  • Audio input can also be provided in the form of an audio file

  • the audio input sent to the Google API has to be a bytestream

  • the audio length for one request can be maximum 1 minute, as per https://cloud.google.com/dialogflow/quotas#es-agent_2

• Required actuators: None

• Required service(s): stream_audio

or audio file

...

audio length + audio type (bytestream)

The following drivers need to be running if testing locally

...

: computer-robot, computer-speaker, and computer-microphone.

Output

The output solely depends on your project and the set-up of your intents and entities of the Dialogflow agent.

The output consists of a dict:

{'intent': '[YOUR_INTENT]', 
'parameters': 
    {'[YOUR_PARAMETER]': '[PARAMETER_RESPONSE]'}, 
'confidence

...

transcript - string

...

entities + their type

...

': [CONFIDENCE_VALUE], 
'text': '[RESPONSE_TEXT]', 
'source': 'audio'}

• intent: str

  • the intent on which the audio was recognised, corresponding to the intent set on the agent

• parameters: dict

  • the parameters defined in the agent

  • each parameter is a str key, with the its response as str value pairing

• confidence level: int

  • number ranging from 0 to 100 that defines how confident the API is with the intent and text detection

• text: str

  • speech-to-text response from the API

• source: str

  • for the SIC framework Dialogflow usage, the source is always ‘audio’

Parameters

1. Dialogflow Keyfile path - string: str

2. Dialogflow Project ID - string: str

The parameters are set at BasicSICConnector instantiation time.

Service Configuration

Our service communicates with a Dialogflow agent to achieve its intended purpose, and it does so by using a project ID and a key file, however, if . If you happen to have them, you may skip this section.

...

In order to use our service for your purposes, there are two classes with which you must interact, namely BasicSICConnector and ActionRunneran instance of the BasicSICConnector class has to be created. You can find the details of these classes this class here. You may also need a class to manage speech_recognition attempts and a callback function for retrieving a recognized entity from the detection result.

...

1. You have the correct agent name and keyfile ( path) as parameters for an instance of the class Example in the example file and are passing it as parameters when creating an instance of BasicSICConnector.

2. You have the Dialogflow services running and you have the relevant local devices running.

...

The following file, https://bitbucket.org/socialroboticshub/connectors/src/master/python/speech_recognition_example.py, is available for the purpose of demonstration. Two questions are dealt with in this example, the . The first being is an entity question where the point of interest is the name of the user, and the second being . The second is a yes, no, or don’t know question.

...

Setting up the agent

In order to deal with the first question, an intent needs to be set up. Intent is something you want to recognise An intent is a value recognised from an end-user, in . In our example that would be , the name of the person. The following steps will set an intent of your Dialogflow agent:

1. Begin by naming your intent, you can name it whatever you wish; we have named it ‘answer_name’.

2. In the section Action and parameters, you should name the intent to what you will use in the program. In the given example, we have used the name ‘answer_name’.

3. In the section context, this section indicates that one is looking for a particular intent. Match the (input)context with the name of the intent, ‘answer-name’ has been assigned in this particular example. Thereafter, set the output(context) to 0, this option is set at 5 by default.

4. In the section Training Phrases, there are empty input strings, which when filled with expressions you would expect one to express, Dialogflow will learn from these to make a model which would also include phrases similar to the list of expressions you gave. Our example has used a list of phrases that you can find in figure 1.

5. Select a word from an expression as a parameter by double-clicking on it and selecting the appropriate entity from the list. It will then automatically appear below ‘Action and parameters' as well; the ‘parameter name’ there will be passed in the result (we use 'name’ here).Navigate to the agent’s page to set the intent, training phrases and parameters.

6. Create an agent intent.

   • It is recommended that the name suggests the kind of answer you are looking for in the audio. In our example, the name of the user (‘answer_name’).

   • the intent defined in the agent should correspond to the intent used in the code

7. Create a context.

   • the number next to the context corresponds to the number of responses expected from the user in that context. In our example, that number is 0

8. Create training phrases for the intent

   • the training phrases should be input examples that contain the intent. In our example, 'my name is name`

   • Dialogflow learns from these phrases and matches future user inputs based on them

9. Create parameters for the intent

   • select words from the training phrases as parameters by double-clicking on them, then match them with their corresponding entity. They automatically appear in the ‘Action and parameters’ section. In our example, we are only interested in the ‘name’ of the user

Our complete intent example thus looks like this (note: using sys.given-name is usually preferred):

...

With these steps, the intent of your Dialogflow agent would be set up successfully for the example file. All that remains now is to ensure to pass your agent name and keyfile(path) as parameters when creating an instance of class Example.

For each intent that you create, make sure that: abstract away + concrete example

Events

...

Events

• onAudioIntent

  • a new intent is detected

• IntentDetectionDone

  • a new intent has finished being detected

• onAudioLanguage

  • the audio language has been changed

• LoadAudioDone

  • if an audio file is used, the event is raised when the file has finished being loaded

Known Issues

• There is a rare bug where sometimes Dialogflow will suddenly only respond with ‘UNAUTHENTICATED’ errors. Restarting Docker and/or your entire machine seems to be the only way to resolve this.