Dialogflow

Do you want to get to know a very cool Google Cloud tool, and like to optimize your conversational agent’s understanding of what a user says? You will create intents with training phrases, and entities in order to provide your agent with the vocabulary and natural language understanding capabilities it needs to converse about the recipes it will be able to recommend. What is important to know for this project is that Dialogflow provides the core Natural Language Understanding (NLU) capabilities that your conversational agent will use to interact with its users. NLU here means that user utterances are transcribed into text, these texts are classified into intents, and the things a user talks about and are useful for recommending recipes are extracted from that text using so-called entities. The Dialogflow agent that you will create thus is an NLU agent. It will be connected to a MARBEL agent which receives the transcripts, intents, and entities from the Dialogflow agent. The MARBEL agent uses the information extracted by the Dialogflow agent to manage the dialog with a user, using conversational patterns to structure the interaction (see below). The MARBEL agent thus is a dialog manager. We have provided you with a simple MARBEL agent to get started. The first thing you need to do is create your own team’s Dialogflow agent. One of your team members should create such an agent and share it with the other team members. Your team only needs one Dialogflow agent!

Run the code!

You now should be able to run the code. Try it by following the instructions here: Run your Conversational Agent. What do you see? Why are you seeing this? Inspect the html.pl file to understand what has happened.

A Pattern to Get Started

The dialog manager agent uses an agenda to structure the conversation with a user. An agenda consists of one or more conversational patterns; conversational patterns are templates for small sub-dialogs or mini-dialogs that can be viewed as building blocks to create a larger dialog or conversation. By inspecting this agenda, the agent can figure out what it should do and in what order things should be handled in the conversation. The agenda is managed by a MARBEL agent, which implements the dialog manager. There already is some code for initializing the agenda in the dialog_init.mod2g module. Open this file and navigate to the code line where the agenda is inserted into the agent’s database (state). As you can see, the agenda in the agent we provided to you as a starting point is still empty (it is the empty list []). We will use some of the patterns that we will create in the project to define the agent’s agenda by adding them to the list. To get started, we need to add something already to the agenda. We can use the start pattern for this which has already been defined in the patterns.pl file (check it out). This pattern just waits for the user to click a button on the screen. Below we will create such a screen where a user can click a start button to initiate the interaction. For now, let’s just add the start pattern to the agent’s agenda:

Visuals

The conversational agent that we will create not only talks but also shows things using dynamic web pages that are displayed on a screen. These dynamic web pages will provide the user with some additional visual support to be able to keep track of some of the things that happened in the conversation. We want the agent, for example, to display information about where we are at in the conversation by, for example, displaying subtitles that refer to different parts (conversational patterns) of the conversation. We will also want the agent to display the preferences for or constraints on recipes that have already been added by the user and show how many recipes still fit those preferences. This will help the user understand what the agent is doing and help them remember which preferences they already indicated to the agent.

The instructions that we will give you will guide you on how to implement quite a basic visual design. Thereafter it is up to you to make things look better and make additional design choices of your own. Creating the basic visual design should not require too much effort and should leave you with plenty of opportunity and time to extend and experiment with your own visual design choices!

The code for generating HTML is provided in the Prolog html.pl file that is part of your MARBEL agent project. Open this file to inspect it. When you scroll through the file, you can see that the file is organized into four parts:

1. Page layouts: Here you should put all the code for the various webpage designs that are introduced with the capabilities you need to implement. You should add code for the start screen right at the top of this part of the file. The place to put this code is already indicated by a code comment for your convenience.

2. Code for generating HTML elements: This part provides Prolog definitions for generating elements of an HTML page. One of the first rules in this section defines the button(Content, Html) predicate. The clause defining this predicate first collects some template code for creating a simple button element in HTML (the template sits right above the defining clause in the file) and then applies this template using the Content parameter to generate a button that says whatever you put in Content. We will use this predicate to add a button on our start page below! As you can see in the file, there are predicates for adding several other types of content to the HTML page, including, for example, lists of items, and images. To better understand the definitions in the html.pl file, and for more background on how to create HTML code using Bootstrap read theVisual Support Guide.

3. Code for generating the content for the body of an HTML webpage: This part defines a predicate html(Content, AddHeader, AddFooter, Html) for generating the main HTML code and when indicated also a header (when AddHeader is instantiated with true) and/or footer (when AddFooter is instantiated with true). The part also introduces various other useful predicates for facilitating the interaction with a user, e.g., for enabling audio interaction on a webpage and displaying a microphone icon.

4. A helper predicate: A dynamic predicate pageUpToDate(Name) that will be used by the agent to keep track of whether a page that is displayed is up to date or not (and needs updating in that case).

Start Page

So let's get started! The first page that we will implement together is a start page. We will provide a detailed step-by-step guide for you on how to create it. Pages are defined by Prolog rules. The Prolog rule that we will code here for creating a start page will also serve as a kind of template example of how you can create the other pages using Prolog, HTML, and Bootstrap. The basic idea is to define a predicate page(PageName, Text, Html) which generates HTML code (a complete webpage) that is returned in the variable Html for a page named PageName; the idea behind the Text parameter is to use it for adding specific text to a page.

Each of the pages that we will create is connected to a specific conversational pattern. Most of the patterns for these pages will still need to be implemented as part of later capabilities too. But the initial pattern linked to the start page that we will add here already exists and is the start pattern that we already added to the agenda above. As a convention, we will always use the name of a conversational pattern as a name for a page that we create too. There is some logic behind this choice that we will explain below. For the start page, we therefore need to specify a rule that defines the page(start, _, Html) that will generate HTML code for a page called start.

The main requirement for the start page is that it has a button the user can click when they are ready to start a conversation. The basic idea thus is to display an HTML page at the start before a conversation has started that allows a user to start the conversation with a button click. The start page can also be used to provide some information about the agent before a user starts talking to it. The basic page layout that we will create consists of four parts: a title, some introductory text, an instruction text on when to press the start button, and, last but not least, the button itself that says “Start”.

The Prolog rule that we will create for the page(start, _, Html) has a basic template structure that we can reuse for most other pages too. It always 1: starts with checking that the active conversational pattern (at ‘top level’, but disregard that for now) has the same name (pattern ID) as the page name by using the currentTopLevel(PatternID) predicate; in our case, here we replace PatternID with start. Only if that is the case, the rule will succeed, and code for a complete HTML page will be generated. After performing this check, 2: Prolog code for constructing HTML code for the page follows. We typically organize pages in rows (often using matching Bootstrap row elements) each of which matches with one of the three parts of our page layout (title, text, button). This way of organizing our HTML pages provides a generic kind of template approach to structure any page that can be easily reused. Then 3: we use a built-in predicate atomic_list_concat(+List, -Atom) for putting the different parts (rows) together. And, finally, 4: we use a predefined predicate html(Body, false, false, Html) for generating a Bootstrap-compliant HTML page without a header and without a footer. The overall structure of our start page code then will look like this:

 %%% Page layout for start page (before conversation has started).
 page(start, _, Html) :-
    % 1: Condition for when to show this page.
		currentTopLevel(start),
	% 2: Constructing the HTLM page.
	% First row: a warning inside a Jumbotron element.
		CODE WE STILL NEED TO FIGURE OUT, SEE BELOW.
	% Second row: introductory text inside a Jumbtron element.
		CODE WE STILL NEED TO FIGURE OUT, SEE BELOW.
	% Third row: instruction and Start button inside an alert.
		CODE WE STILL NEED TO FIGURE OUT, SEE BELOW.
	% 3: Putting everything together.
		atomic_list_concat([FirstRow, SecondRow, ThirdRow], MainElementContent),
	% 4: Create the HTML page.
		html(MainElementContent, false, false, Html).

Note that in the above code template at line 15 three variables FirstRow, SecondRow, and ThirdRow have been used to refer to the first, second, and third row of our HTML code. The atomic_list_concat/2 predicate is used to concatenate these variables (as elements of a list) into the overall MainElementContent HTML code for our page. The predicate concatenates atoms which means that the terms bound to these variables should be atoms too. The next thing we need to do is to complete the code template by writing code that generates the HTML code for each of the three rows and substituting those code snippets, respectively, at lines 7, 9, and 11.

First row. Let’s first add some simple HTML code to create a heading for our page as the first row. We will straightforwardly do this by unifying the first-row variable FirstRow with an atom string containing the HTML code. We use the predefined div/4 predicate to make a https://www.w3schools.com/bootstrap4/bootstrap_jumbotron.asp element and center a large heading text within it.

div('<h1 class="text-center">Please read this first:</h1>', 'jumbotron', '', FirstRow)

Note that a string between single quotes also is an atom in SWI Prolog (if you want to know more see here). For those of you who know basic HTML the string between the quotes should look familiar: the HTML code creates a large heading text using the tags <h1> and </h1>. For those of you for whom HTML is new, you can find out more here.

Second row. For the second row, we will add a few lines of text to a paragraph and embed that again in a https://www.w3schools.com/bootstrap4/bootstrap_jumbotron.asp. We will first concatenate the lines as atomic strings (between single quotes) using the atomic_list_concat/2 predicate, we will then create a paragraph element using this content by using the predefined applyTemplate(Template, Content, Html) predicate and a simple HTML code template for a paragraph element with centered text, and, finally, add that into a Bootstrap Jumbotron element. The three-part code snippet that we are looking for is this:

atomic_list_concat([
    'You are about to interact with our agent <b>YourAgentName</b>.<br>',
	'It can help you find a recipe to your taste.'
], ParagraphContent),
applyTemplate('<p><center>~a</center></p>', ParagraphContent, ParagraphElement),
div(ParagraphElement, 'jumbotron', '', SecondRow)

We have just put some text there for you to get things moving but, of course, feel free to change it in any way you want. The main point that we want to illustrate here is that it is convenient to use the atomic_list_concat/2 predicate to concatenate a list of string atoms (lines) into a single Prolog atom. It helps you keep track and provides a nice overview of the lines of text you want to put on a page. It also avoids potential syntax issues, as using whitespaces and newlines in Prolog code may often cause problems. Check out the utils.pl file for the definition of the applyTemplate/3 predicate and the html.pl file for the definition of the div/4 predicate. The applyTemplate/3 predicate adds the few lines of text in our example code into an HTML paragraph element using the <p> and </p> tags. The <center> tag is used to center the text inside the paragraph. In our code example above, the applyTemplate/3 substitutes the (term bound to the) variable ParagraphContent for the ~a placeholder that is part of the template string. Thereafter the paragraph element is embedded in a Bootstrap alert element using the div/4 predicate. You can find out more on the Bootstrap template for alerts here. Finally, the result is unified with our SecondRow variable.

Third row. Our third row will combine an instruction text with a button with bold Start text inside of a https://www.w3schools.com/bootstrap4/bootstrap_alerts.asp element. We use the predefined button/4 predicate for creating a button element (check out the html.pl file). This predicate will allow us to specify the atom that the agent will receive when a user clicks the button. Because we want to keep things as uniform as possible, we will also use the atom 'start' (which is the same atom that we used for our pattern and webpage names). Check out the https://www.w3schools.com/bootstrap4/bootstrap_buttons.asp page to learn more about some of the class options that we added below.

button('<b>Start</b>', 'btn btn-lg btn-info', 'start', Button)

We want to use a (somewhat smaller than before) heading text for the instruction and show that before our button. We use the atomic_list_concat/2 predicate for concatenating these two elements:

atomic_list_concat(['<h3>Done reading? Then please press start to begin</h3>', Button], 
  ThirdRowContent)

And, finally, we embed this content inside an alert element, with some added classes to style the alert, using the div/4 predicate:

div(ThirdRowContent, 'alert alert-info text-center', '', ThirdRow)

We now can fill in the gaps in the Prolog rule for defining our first page using the three different code snippets for the three different parts (rows) we want to display on our start page. It remains for you to put this all together and add it to the html.pl file in the right place. Just to be sure, when you copy-paste the rows into the overall page template code, make sure to also add the commas that are still missing at the end of each of the three code snippets.

You now have created your first page that your conversational agent can display. The page does not look particularly great yet and there is still much that can be improved. How exactly you and your team organize or create your HTML pages is up to you, as long as you fulfill the minimal conditions that the page must meet. Hence, make sure to revisit this page and revise it again later during the project!

Run your agent again!

Run your agent again, see here for the instructions for Run your Conversational Agent.

You should now see the start page that we created! Don’t yet press the button…

Switch to the Debug perspective in Eclipse, select the MARBEL Debugging Engine in the Debug area, and press pause. Inspect the agent’s state and check out in particular the session/1 predicate.

Now start the agent again by pressing resume. And then press the start button on the page. Go back again to the Debug perspective and pause the agent again. Now again inspect the agent’s state and in particular the session/1 predicate. Double-click on the session/1 fact if you can't see it completely.

What has happened?