Here is an overview of how to implement your MARBEL agent step by step.

1. Getting Started with your MARBEL Agent

Summary Description

The first step is to get your conversational agent up and running. There is one ingredient that is still missing, but required before you can run the code provided to you. You need to create your own Dialogflow agent that you will use throughout the project and connect it to the MARBEL agent that manages the dialog (or conversation) between a user and your agent. When you have created this agent, you will be able to run your agent. Next, you will also be asked to put something in the agenda of the MARBEL agent to get things going. And, last but not least, you need to add code for creating an initial start screen (a simple webpage). When you have done this too, you can run your conversational agent and should see the start webpage that you created. Your agent won’t do any talking yet, though. It is just a first step.

Implementation Tasks Overview

1. Set Up Dialogflow Agent

   • Create a Dialogflow agent with default settings (English as the language).

   • Share it with team members by assigning the Developer role.

   • Import the provided zip file for basic setup.

2. Integrate Dialogflow with MARBEL

   • Download the JSON key file for Dialogflow and add it to the agent folder in your repository.

   • Retrieve the Dialogflow Project ID and update the flowkey and flowagent parameters in the .mas2g file.

3. Initialize MARBEL Agenda

   • Replace the empty agenda in the dialog_init.mod2g file with the start pattern (agenda([start])).

4. Create the Start Page

   • Design a basic start page in html.pl using Prolog and Bootstrap, including a title, introductory text, and a “Start” button.

5. Run and Debug the Agent

   • Test the integration by running the MARBEL agent and verifying state changes in Eclipse’s Debug perspective.

Instructions

Getting Started with your MARBEL Agent

2. Greet, and Self-Identify

Summary Description

Natural conversations are typically opened. Our agent should not just straight away embark on the task of recommending a recipe. It should first secure the attention of its user. Opening a conversation is a basic conversational competence. A common way of opening a conversation is by greeting each other. You should provide your agent with this basic conversational competence too. It should, moreover, be able to introduce itself to its user as in the following conversational pattern. A welcoming webpage should also be created.

Implementation Tasks Overview

Tasks Focused on Dialogue Patterns

1. Implement Basic Greeting Pattern

   • Define the c10 pattern in patterns.pl for the greeting exchange: [agent, greeting] and [user, greeting].

   • Add the condition agentName('') to ensure it is used only when the agent has no name.

2. Add Self-Identification Pattern

   • Extend c10 in patterns.pl to include [agent, selfIdentification].

   • Add not(agentName('')) to ensure it is used only when the agent has a name.

3. Specify Agent Responses

   • Add phrases for greeting and selfIdentification intents in responses.pl.

   • Dynamically retrieve the agent’s name using agentName(Name) to generate self-identification responses.

4. Add Patterns to the Agenda

   • Update dialog_init.mod2g to include c10 in the agenda after start.

Tasks Focused on Visual Support

1. Create a Welcoming Page

   • Define page(c10, _, Html) in html.pl to create a welcoming page.

   • Include a microphone icon for user interaction and dynamically display the agent’s name if available.

2. Design Visual Elements

   • Use Bootstrap components to organize text, images, and interactive elements into sections like greetings and microphone access.

Tasks Focused on Debugging and Testing

1. Test Greeting Patterns

   • Test the basic greeting pattern by running the agent and initiating interaction from the Start page.

   • Set the agent’s name in dialog_init.mod2g and verify self-identification functionality.

2. Test User Response Variations

   • Provide non-greeting inputs to test the agent’s response and inspect session updates in Debug mode.

Instructions

[TBU]Greet, and Self-Identify

3. Request a Random Recipe Recommendation

Summary Description

The second capability will enable the agent to recommend a recipe when a user does not care (to refine) what kind of recipe they are looking for (see the example conversational pattern below). A new pattern will be introduced to this purpose for selecting a recipe, and a new intent should be added to your Dialogflow agent to enable understanding user requests for recommending “just some recipe”. Quite some work will need to be done for introducing the first code needed for retrieving recipes from the recipe database. The first versions of two new pages will also need to be created. One page for making clear to the user that they should inform the agent about features of recipes that help refine the selection of recipes and clarify what the user is looking for. Another page for asking a user to (dis)confirm that the recipe that the agent has selected and recommends to the user is what the user is looking for.

Implementation Tasks Overview

Tasks Focused on Dialogue Patterns

1. Implement Recipe Selection Pattern

   • Define the a50recipeSelect pattern in patterns.pl with these moves:

     • [agent, specifyGoal]: Agent asks the user what recipe they want.

     • [user, requestRecommendation]: User requests a random recipe.

     • [agent, recommend]: Agent suggests a recipe.

   • Add [agent, insert(a50recipeConfirm)] to direct the agent to the recipe confirmation pattern.

2. Implement Recipe Confirmation Pattern

   • Define the a50recipeConfirm pattern in patterns.pl to allow users to confirm or disconfirm a recipe.

3. Specify Agent Responses

   • Add responses in responses.pl:

     • text(specifyGoal, "..."): Agent asks about recipe preferences.

     • text(recommend, "..."): Agent suggests a random recipe.

Tasks Focused on Recipe Filtering

1. Add Recipes to the Database

   • Each team member adds one favorite recipe to recipe_database.pl, ensuring unique recipe IDs and consistent ingredient names.

2. Implement Recipe Selection Logic

   • Add rules in recipe_selection.pl:

     • currentRecipe(RecipeID): Retrieves the selected recipe from memory.

     • recipeIDs(RecipeIDs): Collects all recipe IDs from the database.

     • recipesFiltered(RecipeIDs): Filters recipes based on user preferences or constraints.

3. Update Conversational Memory

   • Add a rule in dialog_update.mod2g to update the memory with the randomly selected recipe when a user requests a recommendation.

Tasks Focused on Visual Support

1. Design Recipe Recommendation Page

   • Define page(a50recipeSelect, _, Html) in html.pl to display:

     • A simple heading asking for user preferences.

     • A microphone button for user input.

2. Design Recipe Confirmation Page

   • Define page(a50recipeConfirm, _, Html) in html.pl to display:

     • The recommended recipe name and image.

     • A microphone button for user confirmation.

Tasks Focused on Debugging and Testing

1. Test Recipe Recommendation

   • Add a50recipeSelect to the agenda in dialog_init.mod2g and verify the agent's ability to:

     • Ask for recipe preferences.

     • Respond with a random recipe recommendation.

2. Test Recipe Confirmation

   • Ensure the agent correctly displays the recommended recipe and allows user confirmation.

Instructions

[TBU]Request a Recommendation

4. Select Recipes by Name

Summary Description

Instead of leaving it up to the agent to suggest a recipe and choose one out of all remaining recipes, a user also can mention a specific recipe themselves and ask the agent to present that. This means that the Dialogflow agent should have knowledge of all of the recipes that are in the database too. We will make this available to that agent by adding a recipe entity (type). That will also enable the agent to recognize these recipes in user expressions.

Implementation Tasks Overview

Tasks Focused on Dialogue Patterns

1. Add a New Recipe Request Pattern

   • Define a variant of the a50recipeSelect pattern in patterns.pl with these moves:

     • [agent, specifyGoal]: Agent asks what recipe the user wants to cook.

     • [user, recipeRequest]: User requests a specific recipe by name.

     • [agent, recipeChoiceReceipt]: Agent acknowledges the selected recipe.

     • [agent, insert(a50recipeConfirm)]: Move to the confirmation phase.

2. Specify the Agent’s Recipe Acknowledgment Response

   • In responses.pl, define a rule for text(recipeChoiceReceipt, Txt) to construct the response dynamically.

   • Retrieve the recipe name from memory using currentRecipe/1 and construct the acknowledgment (e.g., "Artichoke and pine nut pasta is a great choice!") using string_concat/3.

Tasks Focused on Visual Support

1. Update Visuals for Recipe Request

   • Adjust the recipe recommendation page (page(a50recipeSelect, _, Html)) in html.pl to clearly guide users toward selecting a recipe by name.

   • Ensure the page is engaging and displays useful instructions or examples for specifying a recipe.

2. Support Alternate Input Methods

   • Uncomment the chatbox code in the footer/1 predicate in html.pl to allow users to type their recipe request as an alternative to voice input.

Tasks Focused on Debugging and Testing

1. Test Recipe Request Handling

   • Add the new pattern to the agent’s agenda in dialog_init.mod2g after the c10 pattern.

   • Test the interaction flow:

     • Agent asks for a recipe.

     • User requests a specific recipe.

     • Agent acknowledges the recipe choice and moves to the confirmation phase.

2. Verify Intent Recognition

   • Check the Dialogflow Training Tool to confirm the recipeRequest intent is recognized.

   • Verify the SIC server logs show the correct intent and transcript.

3. Handle ASR Failures

   • Use the chatbox feature to avoid ASR failures and ensure the agent can handle specific recipe requests reliably.

Instructions

[TBU]Select Recipes by Name

5. Filter Recipes by Ingredients, Cuisines and Meal Types

Implementing this capability will involve implementing some of the conversational and reasoning competences that a conversational recipe recommendation agent should have. It will enable users to request recipes which use specific ingredient (types) or ask for recipes from a specific cuisine. This one of the capabilities that requires quite a big amount of work. Some of the bigger challenges will be to make Dialogflow understand ingredients (at least those that are in our database; and cuisines, for that matter, but those are fewer), to implement the logic to filter the recipe database with the feature requests of a user, and to present those in a nice way to the user on a webpage.

Summary Description

Implementation Tasks Overview

Instructions

Unexpected Intents

Summary Description

A conversational agent should be able to handle dialog moves of users that do not necessarily fit into the currently active pattern. Such moves are “unexpected” in the sense that they do not fit into a conversational pattern, but should be expected as users will almost always slightly deviate from a rigidly framed pattern. Typical examples are expressions of appreciation and requests for information about how the agent can assist a user (a capability check). A second type of unexpected move is not due to user behavior but due to failures of speech recognition. Dialogflow will match with a so-called default fallback intent in cases where it is unable to recognize what a user says as and cannot classify it as one of the (other) intents of the agent. There is another case that we will look into too where what a user says does not seem to fit into the conversational context. Again, we can capture the “unexpected” in patterns, which will enable the conversational agent to handle them:

Appreciation example pattern:

U: Thanks

A: You're welcome.

Capability check example pattern:

U: What can you do?

A: I can ____.

Repair example pattern:

U: Have you read the Hobbit?

A: What do you mean?

Implementation Tasks Overview

Instructions

Filter Recipes by Ingredients and Cuisine

Implementing this capability will involve implementing some of the conversational and reasoning competences that a conversational recipe recommendation agent should have. It will enable users to request recipes which use specific ingredient (types) or ask for recipes from a specific cuisine. This one of the capabilities that requires quite a big amount of work. Some of the bigger challenges will be to make Dialogflow understand ingredients (at least those that are in our database; and cuisines, for that matter, but those are fewer), to implement the logic to filter the recipe database with the feature requests of a user, and to present those in a nice way to the user on a webpage.

Summary Description

A: What recipe would you like to cook?

U: I'd like to make a recipe with ____ *.

A: All the recipes left include ____ *. Do you want to add another preference?

• insert name of an ingredient (type) or cuisine here

Implementation Tasks Overview

Instructions

Filter Recipes by Dietary Restrictions

Summary Description

We continue to extend the capabilities of the agent to handle user requests. The focus will be to enable a user to ask for recipes that meet dietary restrictions. This will also involve extending the logic of reasoning about ingredients. The main challenge will be to come up with an efficient approach to define this logic.

A: What recipe would you like to cook?

U: I'd like to make a ____ * recipe.

A: Here is a list of recipes that fit your preferences.

• insert a recipe feature such as vegan, for example, here

Implementation Tasks Overview

Instructions

Filter Recipes by Excluding Features

Summary Description

The last extension to increase the capabilities of our agent will look into feature requests that exclude aspects of a recipe. There are many examples of this, but we will focus on user requests that ask for excluding an ingredient (type), for recipes that are not from a particular cuisine (e.g., not from Asia), or do not have a specific dietary restriction (e.g., are not vegan). You are, of course, invited to extend the agent with other similar capabilities to exclude aspects of a recipe.

A: What recipe would you like to cook?

U: I want a recipe without ____ *.

A: Here is a list of recipes that fit your preferences.

• insert an ingredient such as salt here

Implementation Tasks Overview

Instructions

Confirm the Chosen Recipe

Summary Description

When users have finished their search for a nice recipe to cook, their choice should be displayed (which we already implemented for https://socialrobotics.atlassian.net/wiki/pages/createpage.action?spaceKey=PM2&title=2025%20Capability%205%3A%20Filter%20Recipes%20by%20Ingredients%20and%20Cuisine) and they should be asked to check the details to confirm the recipe is indeed what they would like to cook. Of course, we need to take at least two scenarios into account: (1) a user confirms they are happy with the recipe, or (2) they indicate it is after all not quite what they were looking for. If all is fine and well, the agent should close the conversation by saying farewell. Otherwise, the conversation should move back to the recipe selection stage. The following conversational pattern needs to be implemented:

A: Can you confirm ___* is the recipe you would like to cook?

U: Yes. (Alternatively: No.)

A: Great. (Alternatively: That is Unfortunate. [Move back to the recipe selection stage])

• insert name of recipe here

Implementation Tasks Overview

Instructions

Removing Filters and Showing Recipes on Demand

Summary Description

The last capability on our list before we ask you to extend the capabilities of the conversational agent using your own insights and ideas is about allowing users to remove requests they made before and to allow them to indicate that they are done providing their preferences. You should implement the following patterns:

A pattern enabling users to remove a recipe feature request:

U: Could you please remove the cuisine and servings requests?

A: I removed the cuisine and services requests for you.

A pattern enabling the user to express they are done:

A: Can you elaborate on what you're aiming for in your recipe?

U: I don't want to add anything else.

A: OK. Here is a list of recipes that you can choose from. [Alternative: Sorry, there are still too many recipes left to show them all. Please add more preferences.]

Implementation Tasks Overview

Instructions

Extend Your Agent (Make it Nicer)

Summary Description

Extend your agent even further; look at the

page or talk to your TA for inspiration!

Implementation Tasks Overview