Dialogflow

Ingredient (type) and cuisine entities

We now want to move on to providing our recipe recommendation agent with some more serious capabilities for filtering the recipe database in search of a recipe that meets the features that the user has requested. We will begin with developing a capability for filtering recipes in the database using an ingredient or ingredient type. The idea is that a user then can indicate to the agent that they would like to cook a recipe which uses carrots, fish, or something else as ingredient. Before you can create an intent to match user expressions such as “do you have recipes that use fish as ingredient”, you have to create ingredient entities. These entities are needed to be able to specify which parts of these user expressions should be extracted as ingredient (type) entity entries. This is done as part of the intent specification, and the presence of an entity type is used as a feature by the Dialogflow classifier model to recognize intents.

We begin with the creation of an @ingredientType entity in your Dialogflow agent. The idea behind introducing this entity is that a user may often want to ask for a specific type of ingredient such as cheese (or, e.g., meat, beef, fruit) instead of a specific instance such as taleggio, a kind of cheese from the Lombardy in Italy. For example, a user could say something like “I’d like to cook something with cheese”. We have precompiled a list of ingredient types for you that you can upload as a CSV file to your Dialogflow agent. For more information on how to upload and create entities in Dialogflow, please read the Dialogflow: Creating Intents and Entities page. Create the @ingredientType entity using the following CSV file: .

A more challenging entity to create is the @ingredient entity that consists of all ingredients that are used in the recipe database. You should extract all of these ingredients from the recipe_database.pl file. You should be able to find them and automatically extract them by matching with the ingredient(..., <ingredient name>). In other words, using some clever data parsing techniques that you apply to the recipe_database.pl file. Create a CSV file with all the ingredients that you extract from the database that you name ingredient and upload this file to your Dialogflow agent to create an @ingredient entity.

Finally, to be able to filter on cuisinee extend your Dialogflow agent an @cuisine entity. Here is a CSV file you can use to upload the @cuisine entity into your Dialogflow agent: .

Request for recipe features intent

The intent that allows a user to ask for specific recipe features is one of the most important intents for our recipe recommendation agent. The agent will use these features for filtering recipes to narrow down the choices that a user still has from the remaining recipes from the database. These features will include a variety of criteria, such as specifying an ingredient (type) that is used in the recipe. As we will use the features that a user will ask for as filters on the recipe database, we will name the intent addFilter. Its main purpose is to add filters of the form featureName=featureInstance, for example, ingredient=taleggio.

1. Go to the Dialogflow Console and create a new intent. Name the intent addFilter to indicate its purpose of filtering recipes based on user preferences.

2. Add Training Phrases: Add a large variety of phrases that users might say when they want to filter recipes by ingredient, ingredient type or cuisine, such as "I want my recipe to include taleggio" or "Something with garlic" or “Do you have something with cheese?” or “I’d like a recipe from Japan”. We recommend adding many training phrases as there are infinitely many variations of asking for some recipe feature. You should try to cover as many of those as possible to ensure broad coverage, but you do not need to craft them by hand…. You can use any tool you can think of to create as many training phrases as possible.
   Note that when you add these phrases, Dialogflow will automatically detect entities like @ingredient. If it does not do that, select the ingredient word(s) and manually tag it with these as an @ingredient entity.

3. Don’t forget to make sure the box above the table in the Action and parameters section is filled in with the name of your intent! If this is not already the case, link the ingredient parameter in this section to the @ingredient entity. Make sure that the parameter name is exactly as expected. Do the same for the @ingredientType entity. For both of these parameters, also check the box in the IS LIST column. This should look like:

Prolog and Patterns

We need to refine our logic now for retrieving only those recipes that satisfy specific filters. We already introduced a recipeFiltered/3 predicate for Capability 2: Request a Recommendation but there it did not have to do any serious work yet. We only introduced the base case without any filters (an empty list of filters) but now need to deal with the case where we have some filters that a user provided. The basic idea to define a recursive clause for recipeFiltered/3 is simple: apply the first filter in the list to the recipes and recursively filter the remaining recipes using the remaining filters. We provide you below with the recursive clause that you should add below the base clause that we added earlier in the recipe_selection.pl file:

recipesFiltered(RecipeIDsIn, [ ParamName = Value | Filters], RemainingRecipeIDs) :-
	applyFilter(ParamName, Value, RecipeIDsIn, RecipeIDsOut),
	recipesFiltered(RecipeIDsOut, Filters, RemainingRecipeIDs).

Inspect this rule carefully and make sure that you understand what it is doing.

Checking that a recipe uses an ingredient

If we want to check that a recipe uses an ingredient, we need to be able to check whether the ingredient is included in the ingredient list of that recipe. For ingredient types, things are slightly more complicated. To do both checks, we will add two rules for defining the hasIngredient(RecipeID, Ingr) predicate that succeed if the recipe with identifier RecipeID uses the ingredient (type) Ingr. Both rules need to be added to the ingredient_hiearchy.pl file (at a location indicated in that file).

The first Prolog rule for hasIngredient is designed to determine if a specific ingredient is used in a given recipe. It simply checks if Ingr is included in the ingredient list of recipe RecipeID by using the ingredient/2 predicate (see the recipe_database.pl file). The second rule is designed to determine if a recipe uses an ingredient type. It assumes that Ingr is a type of ingredient (such as meat). To check whether a recipe uses that ingredient type, however, we need to find a specific ingredient such as steak that is an example instance of the type (only those are directly associated with a recipe in the recipe database). Another example would be apple as an instance of the type fruit. We do not have to specify all of these relations as a lot of the work has already done for you and you can use the typeIngredient/2 predicate in the ingredient_hiearchy.pl file (check out the typeIngredient/2 facts in that file). Now use that predicate in combination with the ingredient/2 predicate to define the second rule. Add your rules to the ingredient_hierarchies.pl file.

Applying filters: Ingredient, ingredient type, and cuisine

After you add the recursive clause for recipesFiltered/3 above, you will see that Eclipse will start to complain that the applyFilter/4 predicate has not been defined. We have reduced the problem of filtering recipes to the problem of applying a single filter to a set of (remaining) recipes. The idea is that the applyFilter(+ParamName, +Value, +RecipeIDs, -FilteredRecipes) filters the RecipeIDs provided as input by means of the feature ParamName using a specific Value and returns the recipes that satisfy this feature in the output argument FilteredRecipes. The parameter name ParamName refers to a feature that a recipe should have, for example, it should be of a certain cuisine. The specific value Value of the recipe feature specifies what exact feature request is made, for example, the cuisine should be Chinese. For each feature we need to write a separate applyFilter/4 rule. For now, we need to provide a rule for the ‘ingredient' feature (use the same name for the feature as the name of the corresponding parameter name used in the addFilter intent in Dialogflow), the 'ingredienttype' feature, and the 'cuisine' feature. The bodies of the first two rules can be designed as exact copies of each other when we use the hasIngredient/2 predicate (which basically hides the difference between an ingredient and an ingredient type). You must find all (Hint for what Prolog built-in predicate you should use) recipes from the list that you start with that satsify the filter and return these recipes in the output argument list. Add your rules to the recipe_selection.pl file at the location indicated in the file; there you will also find the heads of the rules that you need to define. The rule for filtering on cuisine is very similar too but instead of the hasIngredient/2 predicate you should use the cuisine/2 predicate (see the recipe_database.pl for examples).

Requesting a recipe feature

Looking ahead, we would like to be able to conduct conversational interactions like the following example:

A: What kind of recipe are you thinking about?

U: I’d like a recipe from Japan.

A: The remaining recipes are of Japanese cuisine. Is there anything else you'd like?

U: Recipes that use salmon.

A: All the recipes left include salmon. Do you want to add another preference?

U: Teriyaki salmon.

A: Teriyaki salmon is a great choice!

A: Can you check the recipe and let me know if you want to cook this?

Let’s inspect this interaction in more detail and analyse how it is organized. In Moore and Arar’s taxonomy, the parts of the dialog in this example where the user adds a feature request in the second and fourth move can be classified as Pattern A2.1: Open Request. We will use the a21featureRequest label for this pattern and define different variants of it below. The agent initiates with an inquiry about what the user is looking for. This move is part of the a50recipeSelect pattern, a top level pattern that we added to the agent’s agenda. We previously defined variants for that pattern for Capability 2: Request a Recommendation and Capability 3: Select Recipes by Name. The user responds by informing the agent that they are interested in a particular cuisine (Japanese). This is a feature request that we want the agent to manage by means of a new a21featureRequest pattern. The idea is that this pattern is inserted while the user and agent are still performing the top level a50recipeSelect pattern. The a21featureRequest pattern is inserted as a subdialog, which we indicated in the example above by adding indentation. The response of the agent is an acknowledgement move combined with a renewal of the question whether the user wants to add any other feature. We’ll take it that this move is also part of the a21featureRequest pattern and ends this pattern. The fourth user move (a second feature request for the an ingredient type pasta) and the fifth agent move repeats the same a21featureRequest pattern. The interaction concludes with a move of the user mentioning a specific recipe title (Teriyaki salmon). That last move and the agent move following it are both part of the a50recipeSelect pattern we specified for Capability 3: Select Recipes by Name. That pattern added the a50recipeConfirm pattern into the current session of the agent. This pattern asks the user to check the recipe (while showing all relevant details). You still need to add this pattern to implement the capability we are working on right now below.

From a visual (webpage) point of view, we want to inform the user about the progress that is made. The main difference that we think is relevant here is how many recipes are still remaining. That is, how many recipes satisfy all of the user’s requests. The idea here is that it is not useful to show titles and pictures of a large number of recipes (there would be too many for the user to look at) but to show titles and pictures when the number of remaining recipes has reduced to a sufficiently small number (as a design choice, we have chosen <16 here). Assuming that there are fewer than 16 recipes left that are Japanese and use salmon, at the sixth move in the example above, the user can have a look at the displayed recipe titles that are still left and make a choice. That should also clarify how at that move the user can mention a specific recipe title (they see the titles that are left). After selecting a recipe, the agent acknowledges the choice and asks the user to confirm they would like to cook the recipe. At this stage, the conversational context is the a50recipeConfirm pattern and the agent should display all relevant details such as instructions and ingredients that help a user make this decision on the corresponding page.

There is still one important design choice that we need to make at this stage. The choice is whether we want a user to still be able to make feature requests when they are already checking a specific recipe. That is, when the top level conversational context is the a50recipeConfirm pattern and a corresponding page is showing a specific recipe title, picture and other details, should a user still be able to add or remove feature requests? We will proceed with the design choice that a user should be able to still make feature requests when already checking a recipe (but you should feel free to change that if your team disagrees with this choice; you would end up with a different set of patterns than the ones we present below in that case). The consequence of our choice from a conversational point of view is that when either the a50recipeSelect or the a50recipeConfirm pattern are the top level patterns, we should enable the insertion of a feature request pattern as a subdialog in the conversation. Taking our design choice into account, we end up with 2 main variants of the feature request pattern:

1.  A variant for when a user requests a feature while already checking a recipe and the context is a50recipeConfirm. That is, the user already has made a choice (e.g., Teriyaki salmon), is looking at the recipe details for this choice , but still adds a feature request. We assume that the user is still undecided if they make such a move, and will want to move the conversation back to the a50recipeSelect pattern stage by inserting that pattern back into the session at the end of the pattern a21featureRequest.

2. A variant for when a user requests a feature while the context still is selecting a recipe (a50recipeSelect).

Concluding, either way, when a user makes a feature request, we make a design choice to move the conversation to go back to (option 1) or stay (option 2) in the recipe selection stage (i.e., the a50recipeSelect top level context). Implicitly, we are also saying here that making a feature request dialog move is an out of context intent (see Capability 4: Handling Unexpected Intents) when the conversation is neither in the recipe selection nor confirmation stage.

Apart from this design choice, there are a few other aspects of the filtering process that we should take into account. One aspect that we need to consider is whether the user requests are consistent or not. If a new user request conflicts with some of the features that have been requested before, we want the agent to remove those that are conflicting with the new request(s). Here we assume that newer requests should override the older ones. A second important aspect is the distinction between the case where after adding a new filter there are still recipes that satisfy all feature requests versus the case where there are no recipes left that meet all requests. For the latter case, we want the agent to respond differently and instead of an acknowledgement move make a dialog move that informs the user that there is no recipe that meets all feature requests.

Implementing the a21featureRequest pattern variants

To implement the a21featureRequest pattern in the patterns.pl file, we need a few key ingredients to implement the design choice made and the other design aspects that we discussed:

1. First, we want the pattern to be available only when either the a50recipeSelect or the a50recipeConfirm is the top level pattern. We should add this as a condition to the rule defining the pattern similar to how we added the agent name condition to the greeting pattern (see Capability 1: Greet, and Self-Identify).

2. Second, we want to remove any conflicting feature requests. You can use the special action removeConflicts(Params) and insert it in the pattern as a move of the agent. This action is defined in the dialog_update.mod2g file; check it out to better understand what happens. Of course, we need to collect the parameters Params somehow to tell the agent which feature requests (called parameters here) should be checked for conflicts. Use the getParamsPatternInitiatingIntent(user, addFilter, Params) query for this and add it to the condition of each of the rules you implement for the a21featureRequest pattern variants. The predicate is defined in the dialog.pl file; check it out to better understand how it works.

3. Third, we need agent intent labels for the two cases that we can end up: we suggest using ackFilter for acknowledging there are still recipes that meet all requests, and noRecipesLeft when there are no recipes left. The logic that we need to implement to make these moves work differently will be implemented in the responses.pl file below.

You should now also have gotten an idea of the overall structure of the sequence of moves that should be implemented for all feature request pattern variations:

• The pattern starts with an addFilter intent of the user.

• Then the agent should remove conflicting filters using the special action removeConflicts(Params).

• Then the agent should make either acknowledge with an ackFilter move or make a noRecipesLeft move to indicate that there are no recipes left. Optionally, you could also add a featureRemovalRequest move after a noRecipesLeft move to ask the user to remove one of the features again (we’ll get back to this later).

• And, finally, if the top level context is a50recipeConfirm, then the agent should perform the special action insert(a50recipeSelect) to move back to the recipe selection stage.

In total, you should be implemented four variants of a21featureRequest pattern for the different conditions: 2 main variants x 2 cases.

Implementing agent responses

We need to create responses in responses.pl for all the agent intents in the patterns that we added to the patterns.pl file: ackFilter, noRecipesLeft, and featureRemovalRequest.

For the ackFilter intent, we provide one complete example that you can add to the responses.pl file:

text(ackFilter, Txt) :-
	not(recipesFiltered([])),
	getParamsPatternInitiatingIntent(user, addFilter, Params),
	filters_to_text(Params, TxtPart2),
	string_concat("Here are recipes that ", TxtPart2, Txt1),
	string_concat(Txt1, ". Anything else I should add?", Txt).

The intent ackFilter acknowledges that the filters the user requested have been applied, and there are still recipes left after doing so. The first query in the body of the rule checks the list of filtered recipes is not empty. The second query collects the filters the user requested in the Params argument. The third query is predefined in the dialogflow.pl file; check out this file to understand what the query does. The remaining queries concatenate the various parts of the final textual string response in the output argument Txt.

• You should add one or more variants of the rule to vary the textual responses the agent gives when acknowledging one or more filters have been applied.

For the noRecipesLeft intent, you need to implement a Prolog rule that generates an appropriate textual response for the scenario where there are no recipes left after filtering:

1. The rule should have text(noRecipesLeft, ...) as head. The second argument is the response message you want your agent to say to the user. You can use a simple string and replace the ..., for example, with the message: "I added your request but I could not find a recipe that matches all of your preferences. Please remove a filter". Please add some variants yourself.

2. The body of the rule should check that there are no recipes left (the condition which makes the response above appropriate). The condition should be that the list of filtered recipes is empty. Check the first query in the rule for ackFilter to get an idea on how to do that.

For the featureRemovalRequest intent, you should add a simple text/2 fact. Use as response text, for example, "Can you have a look again and remove one of your recipe requirements?".

Visuals

We want to differentiate what we show to a user depending on the number of recipes that still meet the user’s requests. The basic idea is that we should not show a large number of recipes to a user but we can show recipe details when the number of remaining recipes becomes sufficiently small (we chose <16, see also above). As a consequence, we want to create two different versions of the a50recipeSelect page that we created for Capability 2: Request a Recommendation: one that just shows the feature requests made when there are more than 15 recipes that meet these requests, and another for when there are less that 16 which shows the recipe details (titles and pictures) for all of the remaining recipes.

The first recipe recommendation page

Start by extending the code for the recipe recommendation page that we already created. The first thing we want to do is add a condition for when to show this page. This page should be shown when there is still a long list of recipes that match the requests made thus far by a user. We would like to show this page when the number of filtered recipes is still above 15. Use the recipesFiltered/1, check the length of the list of recipes this predicate returns, and add a condition that makes sure it consists of more than 15 recipes.

You should also add more information to this first a50recipeSelect page. The main requirement is that the page shows all the user’s feature requests or filters. You can collect these from the agent’s memory and use the predicate filters_to_strings/1 for this that has already been defined in the dialogflow.pl file. For styling the layout of the page, use rules defined in the html.pl file or add new ones (see also the Visual Support Guide).

Add a second recipe recommendation page

A simple way to start writing some code for this page is to copy-paste the code you already have for the recipe recommendation page and modify it. Most importantly, you need to make sure this page is only shown if the number of filtered recipes is below or equal to 15 and if the user is still trying to select a recipe.

Continue with changing the layout. The main requirement is that an overview of the remaining recipe titles and pictures should be shown. Check out https://www.w3schools.com/bootstrap4/bootstrap_cards.asp and how we already used a card for the recipe confirmation page for Capability 2: Request a Recommendation.

Your final page should look something like this (just an example, you should be able to easily improve!).

Test it Out

Test it out by Run your Conversational Agent. Try different scenarios where you filter on ingredients, ingredient types, and/or cuisine. For example, try adding requests for the following ingredients one by one in order: ginger, salt, sugar, rice. You should end up with exactly 2 recipes left.

Try to filter by Jamaican recipes and see if you receive the following: