Visual support: general instructions

Adding new page lay-outs, general procedure

Bootstrap is a useful framework for working out a display, but may not be so intuitive in combination with prolog. Hereby a short walk-through.

Let’s consider you would like to display a certain lay-out with three images side-by-side. The first step you would take is to check whether such a lay-out is specified in bootstrap. ‘Card decks’ might be a nice option, so we first look into the code that is involved:

What we want to achieve, is to render a page with the html code, but with a dynamic input of images. The trick is to break this down into fragments. First we want to render a given image. In the html, we see that each of the three images are defined as separate cards. We will now use prolog to return the html for such a card, using a given picture url. The aim is to connect in a single string the static html code and the image url (which differs according to the context of the conversation). Let’s work with the good old Pasta Aglio URL for this example. We will define a predicate ‘imgCard’, with an arity of 1 and an arity of 2. We will give it once as a fact, specifying the html-code:

Note that the html is copied, with a replacement of the contents of ‘src’, in the form of ‘~a’ which is a placeholder for the image url. This is needed in order to dynamically add this information. We will do this with the following rule:

In this rule, the Image variable is the input and holds the url of the image, whereas the Html variable is the output html code for this card. In the body of the rule, the imgCard/1 is queried for the html code we already specified. Then, the Image url is placed at the location with `~a' in this html code, using the format/3 predicate.

We can now use the imgCard term to render each of the three images. For the complete display, we will combine them in the broader ‘card-deck’ option, like in the example html we got from bootstrap. We will do this using a self-defined prolog term, with as input the text, button text and the three images, and as output the html code to render. Within this term, we will start specifying the main template, and then collect the images:

Note that atom_concat is used to append the different html-snippets for each of the images. Format is then used to plug in the html code where the `~a' has been specified in the Template variable (as was done for the imgCard.

The current term will display three given images on the screen. Of course you need to make sure that the right image urls can be displayed during the conversation, which you can do using prolog facts in recipes.pl, which can be retrieved in a rule in the dialog_update and dialog_generation modules (or they can be strictly used in the prolog specification in html.pl; for instructions on this, see Using Prolog rules in the html.pl file below.

The current implementation does not include the text that is spoken by the agent. We want to add this to the display as a separate card below the three images. First we will look up the html code for a standard text card. Then we will add this information in a txtCard term and to the template:

As you can see, the extra card html can just be added in the Template after the card-deck we already specified, and it again includes the ~a variable. There can be multiple of such references in the Template string, which are replaced with proper html code in the third entry of the ‘format/3' term. As you can see, we simply added the T variable to this list, which is properly matched with the second instance of ~a.

Using these approaches you should be able to render pages according to your own design choices. Within the specified html code you can play around with sizes, background colors, text fonts, etcetera. Do check the bootstrap website for more inspiration.

Displaying lists

Displaying lists on a screen is a practical feature when dealing with recipe selection and ingredients / utensils per recipe. Hereby some pointers on how to make this possible.

The first step is again to look for the proper template. The bootstrap list-groups template would make a good candidate here:

The goal here is to render the html code with a list of ingredients / recipes / … as input. We will again break it down, starting with specifying the list items:

The way this is defined will look familiar to you, but how are these list items combined together based in a prolog list? We will add another term for this, ‘myList/2', with as first variable the list of terms, and as second variable the html output:

Note that we are using the myMaplist function to combine the contents of the list with the html-code specified in myListItem. While the ‘maplist’ function is built-in into prolog, GOAL might not recognize it. The solution is to add the concrete code for the function to your knowledge base (either in html.pl or elsewhere) as myMaplist:

Using myList, we can now specify the rendering of a page in a new term with a list of items in the way specified above (given that the list is properly included as input to the term, and the term is refered to properly from any of the GOAL modules).

Dynamically rendering pages

There are several methods for dynamically selecting the right layout for the current dialogue context. Note that there are many contexts that can be distinguished in the dialogue at various levels of abstraction. The most natural contexts in the SUPPLE dialogue management approach are patterns and individual intents. Patterns may match to different ‘phases’ or ‘contexts’ in the dialogue (e.g., recipe selection, ingredient check, etc.) but they and intents also may match with specific capabilities (e.g., capability check, etc.). We highlight two different ways below on how to dynamically select and render a layout that corresponds (belongs) with the current context or capability.

Using rules in the various GOAL agent modules

In the implementations above, particular page lay-outs are rendered based on variable inputs to the prolog rules (e.g.: Txt, Image1, Image2 and Image3 in the given examples). Here we will show where and how the Prolog rule can be queried with input to render the page.

A central module in the GOAL agent for rendering pages is dialog_generation.mod2g, as the display often supports what the agent says. Here you can specify the context during which a particular page needs to be rendered, and collect the needed information to include in the page. The example given in the initial agent is very generic.

It is up to you to build on this structure and specify the proper context and information for each of your pages to display. Imagine there is a particular display that you want to link to each recipe step, displaying the step number. This can be specified in the following way:

Note that multiple intents can be specified for the same type of display by using the ; operator in Prolog. It is also important to consider the dialog_update module for rendering a page in certain contexts where the agent is not talking.

Using Prolog rules in the html.pl file

A second approach is to code multiple rules which use different conditions for selecting a layout in a particular context. The lecture slides for week 3 illustrate, for example, how you can query and retrieve the currently active top level pattern id. This allows you to add a condition that checks for a particular pattern id to a Prolog rule in the html.pl file. Other simple conditions that you can add involve counting the number of recipes that are still available while in the recipe selection phase, etc. The general form of the page layout rules would be something like:

There are still many options to vary and for example you can choose the parameters for the myPage predicate that fit your approach best.