| Table of Contents | ||||
|---|---|---|---|---|
|
As indicated on the Project Background Knowledge https://socialrobotics.atlassian.net/wiki/pages/createpage.action?spaceKey=PM2&title=2025%20Project%20Background%20Knowledge page, the main purpose of providing visual support is twofold: to provide (1) support to a user to reduce their cognitive load (the amount of information working memory needs to process at any given time) and (2) an indication of the progress on the task that has been made thus far. Visual support will be provided by means of a webpage. This page aims at explaining how you can develop code for your MARBEL agent to create such a webpage.
To get a Quick Start into how to create a webpage in your MARBEL agent, you can also start by completing the Getting Your Conversational Agent Up and Running https://socialrobotics.atlassian.net/wiki/pages/createpage.action?spaceKey=PM2&title=2025%20Getting%20Your%20Conversational%20Agent%20Up%20and%20Running task first, and then get back to this page to better understand how you can develop a webpage in this project.
| Note |
|---|
Although visual support is key because of the reasons listed above, it is also only meant to be just that: support that facilitates conducting a conversation with your conversational agent. In this project, you should design conversational interaction to be the primary modality of interaction! Requirement: You should develop a webpage which facilitates conversational interaction; in other words, your webpage design should recognize that speech-based interaction is the primary modality of interaction. As such, you should avoid using input-based interactive webpage elements such as forms, dropdown menus, or other elements focused very much on user input via a webpage. |
Using Prolog to build HTML elements in our agent
It is important to be able to synchronize the conversational interaction and the visuals displayed on the webpage shown while talking. The MARBEL agent manages the dialog, and for that reason also is the logical place to locate the control over the visual elements. This agent uses a renderPage(Html) action in the dialog_generation.mod2g module to render a webpage; when performing this action, the Prolog variable Html therefore should be instantiated with HTML webpage code. All we need to know before we can use this action, is how to create this HTML page in Prolog.
...
| Info |
|---|
All Prolog definitions introduced and discussed below can be found in the |
Generating basic HTML elements with short tags
An HTML page consists of HTML elements. HTML elements consists of HTML tags that are used to organize content on that page. Tags often come in pairs of a start and end tag. Basic HTML tags, for example, are the tags <p>YOUR PARAGRAPH TEXT HERE</p> that defines an HTML paragraph, the <h1>...</h1> defining a large heading, <b>...</b> defining bold text, etc. There are many of these basic tags which, moreover, are also short which facilitates their frequent and easy use. The idea to generate basic HTML code for these elements in Prolog is very simple: Simply use single quotes '...' to generate such an element. The table next lists a few examples:
...
There are many simple, short tags where it is easiest to simply put them between single quotes to create a Prolog atom for generating the corresponding HTML code. Another class of short tags are the single tags such as '<br>' for breaking a line and '<hr>' for adding a horizontal line. These are also most easily generated by simply quoting them.
Generating more complex HTML elements with attributes
Most HTML tag pairs can be simply generated by using a start and end tag and by adding some text to create the corresponding HTML. For example, <button>Start</Button> can be used to generate a simple button saying Start. However, to unleash the full power of the HTML language, it is often useful to modify the type or style of an HTML element by adding additional attributes in the form of name="value" to the start tag.
Making a button
For example, the following HTML code creates a large, colored button using the class attribute and several button class values made available by Bootstrap:
...
When evaluating this clause, we would end up with Html having the button HTML code as its value.
Changing and receiving button events in a MARBEL agent
If you would inspect the html.pl file and search for the button predicate, you will find a button/2 predicate too. You will also see that both the button/2 and the button/3 predicates actually are defined in terms of a button/4 predicate button(Content, Class, DataValue, Html). This predicate allows to set the value send as a percept to the agent when the button is clicked. The button/3 definition by default sets this value equal to the Content value displayed on the button. That, however, is not that useful if the content itself consists of HTML tags again. For example, button('<b>Start</b>', 'btn btn-secondary btn-lg', Html) would display a bold text Start but would also send the tags as a percept of the form answer('<b>Start</b>') when the button is clicked. To avoid that, we can use button('<b>Start</b>', 'btn btn-secondary btn-lg', 'start', Html) which would send the more plain and easy to process answer('start') to the MARBEL agent.
Besides the class, style, and data-value attributes there are other global attributes in HTML. Because we think most of these have limited used we have not included these in the code provided to you. You are free, of course, to extend the Prolog code to be also able to use these other attributes.
Other tag examples: ul, img, div, span
We introduce a few other tags and discuss how HTML code for these tags can be generated using some of the predefined predicates in the html.pl file. These examples should provide you with a general understanding of how you can generate HTML code for a variety of HTML elements.
ul tag
Displaying lists is a practical feature when you want to show a recipe and its associated ingredient list. The most basic way to do this is to use the unordered list ul and list item li tags. Here’s a simple example of what the HTML code would look like:
...
This combined query will generate our example HTML code and return it as value in the Html variable.
img tag
Although this project is not about webpage development and is definitely not about promoting fancy interaction (remember to prioritize conversational interaction throughout the project!), by simply adding some images you can bring a webpage to life, making it more appealing to look at while also adding useful functionality. To start with the latter, users would like to see what the result would look like for the recipe they finally choose. To this end, links to the pictures of all of the recipes included in the recipe database have been included for you to make good use of (check out the recipe_database.pl file). But there are many other opportunities, of course, to add pictures if you like to your webpage design. To facilitate this, we have defined the img/4 predicate img(Source, Class, Style, Html) which allows you to create an img HTML element that looks like:
...
Go to the https://www.base64encode.org/ site (don’t change any options on that page)
Go to the Encode files to Base64 format section
Upload an image (note that file size cannot exceed 192MB!)
Press the encode button, and
Download the code as a text file.
Copy the text in the file.
Add a fact with a predicate name for your image of your choosing (to illustrate we used
myImageName) by adding the following template to thehtml.plfile:myImageName('data:image/png;base64,...')Paste and replace the
...in the Prolog fact with the text you copied from the file (a very long encoded string representing your picture)Now you can use
myImageName(Source)instead of thepicture(RecipeNr, Source)query to include your picture on your webpage
div tag
The div tag is a versatile block element that can be used to organize and style HTML content in various ways. In the html.pl file provided to you we have defined a div/4 predicate div(Content, Class, Style, Html) that facilitates the generation of div elements with class and style attributes that you can set. The code that will be generated looks simply like a basic div element without any content yet:
...
| Info |
|---|
Padding and margins The HTML code above uses |
span tag
The span tag is an inline element that is commonly used to style HTML content. We have added a span/3 predicate span(Content, Style, Html) that just does that and nothing else. An example of a span tag that can be useful is to style the content and change the font family that is used within some HTML content:
...
Check out the html.pl file for more predefined Prolog predicates for HTML tags that you can use.
Generating an HTML webpage
Above we have illustrated how to generate individual HTML elements but not yet shown how to generate a complete webpage using one or more of those elements. We will show in this section how you can generate such a webpage and make the agent render it.
...
In this project, we generate the HTML code for the body of an HTML webpage (that is what the template for a page/3 rule above is supposed to do). The head of the webpage (a container for metadata, not to be confused with the header of or headings on a webpage!) is predefined and cannot be modified by our MARBEL agent.
Check out the code for the header/2 and footer/2 predicates in the html.pl to see what HTML code is generated for the header and footer. You are free to change these too if you like, but you should make sure that the microphone icon always is displayed on the relevant pages so a user can engage in talking to your conversational agent (by making it open their microphone and start listening to the user).
Example Prolog Rule for a Page
As indicated above, the idea is to add a rule defining a page(+PatternId, +Txt, -Html) for each conversation pattern that might be ongoing at the top level during the conversation. You will be asked to add a range of different patterns in the patterns.pl file during the project. One of those hinted at above is a greeting pattern with a pattern ID code c10. We will now provide a simple example of rule that generates a page that will be shown when the pattern c10 is active (or ongoing):
...
Now suppose that the query currentTopLevel(c10) succeeds. In that case the page(c10, _, Html) query will also succeed (why?). Suppose, moreover, that our conversational agent is called Cookpanion (you think of a good name for your own agent!) and the query agentName(Name) succeeds with Name='Cookpanion'. What HTML code will then be generated (and bound to the Html output argument variable)? To answer this question, you need to understand how the applyTemplate query works (more on that you can find at the end of this page: https://socialrobotics.atlassian.net/wiki/spaces/PM2PCA2/pages/22642360332709488689/Visual+Support+Guide#Defining-new-predicates-for-generating-HTML-elements). But for now the comment just above the query will be sufficient to piece things together. We get the HTML code <h1>I am Cookpanion</h1><br></center> that binds to IntroHeading. The atomic_list_concat query add this to the end of the <center><h2>Hello!</h2></br> HTML code, which then is used as the content for the alert class using the div predicate. By inspecting the html/2 predicate (check out the html.pl file) we can then figure out that the resulting HTML code will look like this:
...
Perhaps the code that is generated is a bit surprising and it is hard to locate the (just!) three lines that the page(c10,_,Html) query generates (somewhere in the middle, as main part of the page). The code illustrates that the html/2 predicate also does quite a bit of the work for generating the complete body of our webpage, including the generation of a header and footer, and a line at the top to make sure the browser will allow for audio interaction.
Adding a chatbox to your page layout for testing purposes
You will soon enough learn that using speech as the main interface for interaction can be challenging, adding delays and misunderstandings, depending on factors such as the context where it is used (a noisy environment makes ASR much more challenging and may lead to failures), perhaps the accent of the speaker, and other related factors. For testing purposes, it therefore may be useful to also be able to test without these factors potentially complicating the interaction. To this end, we have made a chatbox element available too which makes text-based interaction available as an alternative to (just) speech-based interaction. To add this element to your webpage, we suggest you add the following HTML code to the footer of pages:
...
| Note |
|---|
Although we provide the chatbox option for testing purposes, it is up to you and your team to make sure the agent works also without the chatbox option. The chatbox should not be enabled in the conversational agent code that you submit as final deliverable! |
Using and Understanding the code in html.pl
We illustrate how the predicates introduced above can be used to create more complex HTML code with a card deck example. We also briefly discuss how the Prolog clauses for these predicates have been defined.
A Quick guide to using Bootstrap components not defined in html.pl
We briefly discuss an example to illustrate how to use the code in html.pl yourself with Bootstrap components that you might want to use that have not yet been introduced. To that end, suppose that you would like to create a lay-out with three images side-by-side. The first step you could take is to check which components Bootstrap offers to create such a lay-out. It turns out that a card deck might just be what you were looking for. It is advertised as something you would use when you “Need a set of equal width and height cards that aren’t attached to one another? Use card decks.” Bootstrap also provides the HTML template code as an example of how to use it. This is what that looks like:
...
For creating the card, we also need an image (again we do not bother to fill in the ... in the example, which we leave up to you; check out the https://socialrobotics.atlassian.net/wiki/spaces/PM2PCA2/pages/22642360332709488689/Visual+Support+Guide#img-tag section above on how to do that). We use the img(+Source, +Class, +Style, -Html) predefined predicate and use CardImage as output variable:
...
The HTML code and query above, of course, will not work yet as we used three dots ... for the src attribute. You will still need to make sure that the right image URLs (or Base64 codes) are used to replace these dots and modify the accompanying text in the card bodies to match with these images.
Defining new predicates for generating HTML elements
When you inspect the code in the html.pl file, you will see that a returning pattern has been used to define Prolog predicates for corresponding HTML tags. The general pattern for a tag named tagname looks like this:
...