Team Contribution

Like in week 1, this week requires your parallel contribution to make your cooking assistant work better. Therefore, in this week you will extend on the conversations that your chatbot agent can have about the recipes, diversify the ways in which the user's intents may be worded, and implement several types of fallback, for example, in case the agent does not understand the user's utterance. As a result, your chatbot will be able to:

• Learn how to use features to select a recipe.

• Learn how to communicate about recipe-related information. For example:

  • Recognize the ingredients needed for a recipe that are mentioned by a user and check if the user has all ingredients needed

  • Recognize the cooking utensils needed for a recipe that are mentioned by a user and check if the user has all tools needed for making a chosen recipe

• Switch between recipes

• Perform specific repairs in case of misunderstanding

Recipe Selection

For this week, your main focus is to select a recipe by using the selected feature(s). Moreover, you should check the required ingredients and utensils for a recipe to successfully execute the instructions related to a recipe.

Enable Selection of Recipe by One Feature

Let’s consider the country of origin as a feature to select the recipe. For example, when a user asks ‘I want to cook an Italian dish’. Your cooking should be able to shortlist the recipes which belong to the Italian category. Such a conversation can be developed by means of the following steps:

• The first thing to do is to make sure the agent is able to recognize the category feature. To recognize which category the user is talking about, in your Dialogflow agent create an entity to recognize the country from which a recipe orginates (for example ‘recipeCountry' as entity, with values like ‘italian’, ‘vietnamese’ and 'american’). Also, add an intent like recipeRequestCountry to the Dialogflow agent and integrate these particular entity values by including them in the training examples and marking them. Please follow these for further details.

• The information regarding the country of a recipe needs to be added to the knowledge base. You may add another predicate (like country/2, with the recipe name as first term and the country of origine as second) and use it to add facts to recipes.pl, in order to specify this information for each of the recipes in there.

• To be able to filter for recipes that comply the user’s specified recipe feature (such as Italy as country of origin), add a rule to ‘cooking.pl’ by which you match with the recipes that fit such a query. You may need different rules for different types of features.

• The conversational activity for enabling the user to select a recipe that complies with a certain feature may take different forms. A standard one could be like this:

  • Agent: “Is there a certain cuisine you particularly fancy today?"

  • User: “Do you know any recipes from Turkey”

  • Agent: “These are the recipes that originate in Turkey”

  • User: “Okay, I’ll go for the “Doner Kebab” please!

• Such a conversation needs to be translated to a pattern on patterns.pl. Choose a pattern code (like ‘a5’), and specify the user and agent intents that follow one another for such a conversation.

• The pattern code of course needs to be added to the agenda as part of dialog_init.

• The user intent for specifying the category value (Turkey as country of origin in this example) has already been made. The pattern also includes an intent for choosing a recipe. Add this intent to dialogflow, with recipe name as an entity. Note that this also requires to add the recipe names as entity values of an entity ‘recipe' in dialogflow.

• Also add the agent intents in text.pl. For the second agent intent in the example (indicating that the recipes that match the chosen recipe feature are displayed), it is useful to mention the recipe feature. Inspect the other agent intents (particularly 'recipeConfirm') to see how you can integrate such dynamic information in the text to be spoken.

• Now you should be all set to test this conversational activity. Think of / try out other variants in which such conversations might take place, and add a pattern / intents for these different variants. Note that the different variants typically starts with the agent asking the user for his preference, so the different patterns only need to vary in terms of the subsequent agent- and user intents.

Enable Ingredients Check

We assume that you are already aware of the information on ingredients with their quantities for any recipe in the cooking.pl file. Before starting to provide cooking instructions, the agent should check if the user has all the ingredients needed. We will create a new pattern for doing so and add it to the agenda.

• Like you did for the recipe names, you should now also add an entity called ingredient and list all the ingredients for your recipes (including synonyms) in your Dialogflow agent. We will not use this entity immediately but get back to how we can make good use of it for next week’s assignment.

• Because the user should also be able to indicate that (s)he does not have all ingredients, we will implement a special agenda management instruction, called terminate, for handling such a user response. Instead of the already given restart instruction (see dialog_generation.mod2g) which re-initializes the agent's agenda, the idea of the terminate instruction is to clear the agenda, i.e. make the agenda (an) empty (list).

• You should introduce a new rule in the dialog_generation.mod2g file after the rule for the restart instruction, which implements the terminate instruction.

• When you have implemented the terminate action, continue with creating a new pattern for checking for ingredients with the user. To make them work, you should also add intents in Dialogflow to check the ingredients. You should add several intents to your Dialogflow agent for (dis)confirmation and an ingredientsCheck intent for the chatbot agent. See the patterns.pl file for more instructions.

Enable Utensils Check

It is not sufficient to just check for ingredients. A user should also have the proper cooking utensils available to be able to cook a recipe.

• Add a utensil entity to your Dialogflow agent which is able to recognize all the required cooking utensils. Note that you will need to carefully scan your recipe for all utensils needed. You may also want to check for knives, etc. that might be needed , but are not explicitly mentioned.

• Introduce a new predicate to represent these utensils explicitly and add a pattern for checking with the user if s/he has all the utensils needed. As this part of the assignment is similar to the ingredientscheck we do not provide any detailed instructions for you this time, but assume you will be able to reuse the intents that you introduced before.

• Do not forget to add the cooking utensils check to the agent's agenda!

Recipe Instruction

Sometimes users may want to switch between recipes during instruction. It is also possible that a recipe requested by the user does not exist in your knowledge base. The agent should be able to properly handle such situations. That is, it should be able to switch between recipes, or it should be able to repair the conversation if needed.

Enable Capability Check

The agent should be able to respond to user requests for more information about what it can do. To make this happen, add a C3 pattern to the patterns.pl file. In response to a capability check, the agent should indicate what it can do, and for example make clear which recipes it can instruct by mentioning some of them (typically the three you have chosen).

Enable Switching between Recipes

Sometimes a user may change his or her mind and may want to switch to a different recipe. Although perhaps not the most common action a user might perform, we want to illustrate how easy it now is to allow our agent to support switching between recipes when a user wants to do so while in the middle of an instruction of another recipe.

We will use the powerful option of adding a variant of a pattern to enable a user to switch to another recipe while looking at a recipe instruction for the selected recipe. A variant of a pattern is a pattern with the same pattern id but with a different sequence of actor/intent pairs. Adding variants of a pattern allows the conversational agent to continue with the same pattern type in different ways using the different pattern variants. In particular, it allows the agent to handle different types of responses (intents) that a user might provide to the agent.

• Recall that we introduced a user intent recipeContinuer which enables the user to make clear a step has been completed. We are now imagining that the user will provide a different response, indicating a desire to switch between recipes. The user might, for example, say: ''I'd rather cook risotto'' while being instructed to cook pasta. Note that this will be recognized by your Dialogflow agent as a recipeRequest intent (make sure this is the case!). In order to allow this response and handle it properly, you should now add a variant of the a30recipeStep pattern which expects, instead of a recipeContinuer intent, the recipeRequest intent and handles it by restarting the agenda (use the corresponding instruction).

• Although adding a variant provides a simple tool to handle different user responses, as you will find if you test your agent after adding the a30recipeStep pattern variant, the agent will continue with instructing a recipe step with the next step number, but now for a different recipe! This is not quite what we want.

  • Part of the reason that just adding a variant is not sufficient for what we want to achieve is that the a30recipeStepis a sub-pattern that expands the sequence introduced by another pattern, namely the a30recipe pattern.

  • We need to start a new sequence after restarting to get back to top level! We can do so by using the startNewSequence action and adding it to the rule for restarting in the dialog_generation.mod2g file. This will prompt our conversational agent to ask again for a recipe, but this looks reasonably natural in a context where a user just indicated it wants to continue with a completely different recipe. Try this by adding the action and test if switching recipes now works as expected.

Enable Conversation Repair

You may have noticed while testing your agent, that the agent will get stuck if a user asks for a recipe that is not available. For example, if the user opts for ‘'chicken soup’' for which no recipe is available, and, moreover, Dialogflow is not able to recognize a corresponding entity or intent, the agent will receive a defaultFallback intent (you should make sure that your Dialogflow agent has such a Default Fallback Intent, which by default is available, so do not remove it!).

Our chatbot agent, however, is not able to handle this intent yet. This we will fix now. As a fallback intent may be generated almost always, we will design for a very generic solution to this problem by adding a B1 repair pattern to the patterns.pl file.

A second example where the agent might get stuck is when a user says something that is recognized by Dialogflow but does not match with any of the expected intents, nor with the first intent of any pattern (which then could be initiated). In other words, the intent received by the agent is out of context. Add another B1 repair pattern to indicate that an intent is out of context in the patterns.pl file to fix this.

Visual Support

To improve the understanding of a recipe with respect to different features, you must provide certain visual support.

Visuals related to Recipe Features

We continue with a discussion of our example feature country of origin for searching the recipe database. This feature allows a user to filter by country of origin, for example, by saying ‘I want to cook an Italian dish’. Your conversational agent should be able to illustrate on the screen how such a feature request reduces the number of recipes that match this request in the database. For example, the agent could display the recipe options which belong to the Italian category. The visuals can be an effective means to give the user a glance at available options given the features they have chosen. This can be in the form of specific example recipes, the number of options still left, and/or other types of information that you can think of. As the number of recipes that matches the preference(s) indicated by the user will vary, this is a dynamic problem and your design solution may use different layouts (pages) for displaying the search information depending on the number of options that are still left. For the implementation, you need to address the following tasks to make this possible:

• Write a Prolog rule that collects the recipes from cooking.pl which match the feature(s) chosen by the user. The rule would typically be queried when a user has mentioned a feature category. Keep track of the filters in a separate list (or use the dialogue memory/1 predicate for this).

• Design and implement an html page that visualizes the available options at any point in the recipe selection conversation. Think about the best way to help the user with its decision depending on the number of recipes available. Make sure to include information on these recipes in how the page is set-up (e.g.: you may want to display images or other information for a subset of the recipes (or all of them if there are only a few that fit the filter).

• Add a rule to the dialog_generation module conditioned on the agent intent for informing the user on the options, or design generic html layouts which the agent selects based on the conversational context (e.g., currently active top level pattern), filtering the recipes using the Prolog rule and rendering the html page(s) that you designed.

Visuals related to Recipe Ingredients and Recipe Utensils Check

Visuals can play an important role as well for indicating the utensils or ingredients needed. The ingredient display relates to the section in every written recipe, with a list of ingredients and their required quantity (for a certain number of persons). As this information is potentially rather extended (some recipes may have 20 ingredients), you will need to think of the right balance between detail of information and overview; a screen full of words is not that insightful. Scrolling down might be a solution, but is not ideal as it requires an additional effort from the user. Another option is to show only part of the ingredients / utensils, or simply show them on multiple pages with the user saying next to go the next page. It is up to you to think of the best way to display this information, and choose whether you want to include images of ingredients or just show the ingredients themselves.

Like for the recipe features, you need to collect the right information using one or more prolog rules, design an html-page to present this information in an insightful way to the user and render the display as condition of the agent intent for ingredients check / utensils check by means of a rule you add to the dialog_generation module.

Testing Phase 2

Besides Week 1, Your Cooking Assistant should be able to perform the following:

• Recipe Selection with one feature

• Ingredients Check

• Utensils Check

• Navigate the dialog flow

  • Switching of recipe

  • repair conversations

  • Permit (easy) change of their choice

Week-2 Deliverables

For your Week-2 submission, you need to submit

• Weekly progress report in .doc format (See Guidelines).