Dialogflow → Intent and Slot Classifier with WHISPER
While your intent and slot classifier is in development, you can use Dialogflow, a powerful Google Cloud tool, to handle Automatic Speech Recognition (ASR) and Intent and Slot Classification for your conversational agent. Dialogflow allows you to optimize your agent's understanding of user input by creating intents with training phrases and entities that define the vocabulary and natural language understanding capabilities needed for recipe recommendations.
...
With the replacement in place, your custom system will continue to interface with the MARBEL agent, providing it with the same intent and entity data, but with enhanced flexibility and scalability. This shift eliminates dependency on external tools like Dialogflow, ensuring better alignment with your project’s needs and granting your team full ownership of the conversational pipeline. By transitioning to your classifier and WHISPER, you’ll achieve a more robust, efficient, and adaptive solution for managing user interactions.
Create a Dialogflow agent
Please follow the instructions here: https://cloud.google.com/dialogflow/es/docs/quick/build-agent#create-an-agent.
...
Share the Dialogflow agent with your other team members (assign them the Developer role): https://cloud.google.com/dialogflow/es/docs/access-control.
Use the zip file found in the repository to make an agent follow the steps below to make a Dialogflow agent:
...
Connect it to the MARBEL agent
Create and download your Dialogflow agent’s JSON key file by following the instructions here: https://cloud.google.com/iam/docs/creating-managing-service-account-keys#get-key. Note that you will have to create a service account for the project (button at the top under the blue bar). When you have done this, you can click on the email for the service account. Note that the JSON file is automatically downloaded when it is created.
All group members need to add the JSON file key to your local repository… it will not let you put it on GitHub because it is a key file. Rename the key file to
dialogflow-keyfile.json, and add it undersocial-interaction-cloud/sic_framework/services/eis/dialogflow-keyfile.json
Retrieve the Project ID of your Dialogflow agent: you can find this by clicking on the settings (cogwheel) icon next to your Dialogflow agent’s name here: https://dialogflow.cloud.google.com/.
Open the
.mas2gfile in Eclipse. Then (1) add the name of the JSON key file as the value of theflowkeyenvironment initialization parameter to the mas file, and (2) the Project ID of your Dialogflow agent as the value of theflowagentparameter.
...
You now should be able to run the code. Try it by following the instructions here: [TBU]Run your Conversational Agent. You should see a start page.
A Pattern to Get Started
The dialog manager agent uses an agenda to structure the conversation with a user. An agenda consists of one or more conversational patterns; conversational patterns are templates for small sub-dialogs or mini-dialogs that can be viewed as building blocks to create a larger dialog or conversation. By inspecting this agenda, the agent can figure out what it should do and in what order things should be handled in the conversation. The agenda is managed by a MARBEL agent, which implements the dialog manager. There already is some code for initializing the agenda in the dialog_init.mod2g module. Open this file and navigate to the code line where the agenda is inserted into the agent’s database (state). As you can see, the agenda in the agent we provided to you as a starting point is still empty (it is the empty list []). We will use some of the patterns that we will create in the project to define the agent’s agenda by adding them to the list. To get started, we need to add something already to the agenda. We can use the start pattern for this which has already been defined in the patterns.pl file (check it out). This pattern just waits for the user to click a button on the screen. Below we will create such a screen where a user can click a start button to initiate the interaction. For now, let’s just add the start pattern to the agent’s agenda:
| Panel | ||||||
|---|---|---|---|---|---|---|
| ||||||
In the |
Visuals
The conversational agent that we will create not only talks but also shows things using dynamic web pages that are displayed on a screen. These dynamic web pages will provide the user with some additional visual support to be able to keep track of some of the things that happened in the conversation. We want the agent, for example, to display information about where we are at in the conversation by, for example, displaying subtitles that refer to different parts (conversational patterns) of the conversation. We will also want the agent to display the preferences for or constraints on recipes that have already been added by the user and show how many recipes still fit those preferences. This will help the user understand what the agent is doing and help them remember which preferences they already indicated to the agent.
The instructions that we will give you will guide you on how to implement quite We provide a basic visual design. Thereafter it is up to you to make things look better and make additional design choices of your own. Creating the basic visual design should not require too much effort and should leave you with plenty of opportunity and time to extend and experiment with your own visual design choices!
...
The code for generating HTML is provided in the Prolog html.pl file that is part of your MARBEL agent project. Open this file to inspect it. When you scroll through the file, you can see that the file is organized into four parts:
...
Page layouts: Here you should put all the code for the various webpage designs that are introduced with the capabilities you need to implement. You should add code for the start screen right at the top of this part of the file. The place to put this code is already indicated by a code comment for your convenience.
...
Code for generating HTML elements: This part provides Prolog definitions for generating elements of an HTML page. One of the first rules in this section defines the button(Content, Html) predicate. The clause defining this predicate first collects some template code for creating a simple button element in HTML (the template sits right above the defining clause in the file) and then applies this template using the Content parameter to generate a button that says whatever you put in Content. We will use this predicate to add a button on our start page below! As you can see in the file, there are predicates for adding several other types of content to the HTML page, including, for example, lists of items, and images. To better understand the definitions in the html.pl file, and for more background on how to create HTML code using Bootstrap read the[TBU]Visual Support Guide.
...
Code for generating the content for the body of an HTML webpage: This part defines a predicate html(Content, AddHeader, AddFooter, Html) for generating the main HTML code and when indicated also a header (when AddHeader is instantiated with true) and/or footer (when AddFooter is instantiated with true). The part also introduces various other useful predicates for facilitating the interaction with a user, e.g., for enabling audio interaction on a webpage and displaying a microphone icon.
...
All HTML templates for pages can be found in social-interaction-cloud/sic_framework/services/webserver/templates . Feel free to create them anyways you would like. They do have to include some information. Please see Visual Requirements for specifics.
Start Page
So let's get started! The first page that we will implement together is a start page. We will provide a detailed step-by-step guide for you on how to create it. Pages are defined by Prolog rules. The Prolog rule that we will code here for creating a start page will also serve as a kind of template example of how you can create the other pages using Prolog, HTML, and Bootstrap. The basic idea is to define a predicate page(PageName, Text, Html) which generates HTML code (a complete webpage) that is returned in the variable Html for a page named PageName; the idea behind the Text parameter is to use it for adding specific text to a page.
...
Run your agent again, see here for the instructions for [TBU]Run your Conversational Agent.
...