If you ever baked a cake or cooked a meal from a recipe in a cooking book or on a website opened in your phone/tablet, you probably recognize the hassle of repeatedly checking the details of a step and having to scroll through a screen with hands dirty from cooking. Throughout the Project MAS course, you will be working on the solution to those sorrows: a cooking assistant that conversationally guides the user through a recipe and responds to his/her questions.

Automatic Speech Recognition and Speech Synthesis have come to a proper level for a user's utterances to be (mostly) understood and for the agent to clearly articulate utterances from written text. It remains a challenge, however, to enable an agent to effectively conduct a conversation with the user. Either the agent lacks initiative and responds only to what the user says or asks, or, on the contrary, the agent follows a rather scripted conversation path in which the user has little opportunity to say anything that might change the script. You will be developing an agent that is able to conduct a recipe instruction conversation, where both the user and the agent can take the initiative if they feel the need to. A user might, for example, ask about a recipe step or change to a different recipe, while the agent may ask the user to choose a recipe or provide additional information for answering his/her question. As the course focuses on the dialog management, the interface of the chatbot will be in written text rather than spoken utterances, to bypass challenges like an interruption.

The project MAS course is based upon the basics taught in the Multi-Agent Systems course. This means that you will develop the agent using GOAL and Prolog as knowledge representation language. While the GOAL assignments in the MAS course were linked to the BW4T environment, this course will make use of the Social Interaction Cloud (SIC) environment. In addition, Google Dialogflow is used to handle the speech of the agent, as well as the interpretation of user utterances. The first objective in this course is therefore to set-up and connect those modules to the agent. In the course, you will be working in groups of six, going through the different stages of chatbot development: collecting content, designing conversations, implementing conversation patterns, testing the agent, extending on the content/patterns, and further testing the agent. This will in the end result in the agent program, as well as a report in which you describe its different components and how it was tested. This document will provide a detailed week-by-week description of the targets, while you have the freedom to choose the recipes and extend the agent beyond the basic requirements as far as you like.

Assignment Objectives and Instructions

Once you know how to get started, your main focus is to design and develop a prototype of a Cooking Assistant using the GOAL Agent Programming. You can be as creative as you want, but do not forget to document your code and project.

This assignment requires good communication and contribution throughout the assignment. As per the distribution of work, a group of six will work in three pairs. Thus, the pair working on Recipe Selection (RS) will mainly work on selecting the recipe, while Recipe Instruction (RI) will work on instructions related to the conversational agent design. The Visual Support (VS) team will focus on providing support to the two regarding the related images. Below is a weekly distribution of the MAS project assignment. Click on the week number to get more details related to ‘To-Do’ tasks.

With the new perspective, you might want to consider this flow. Each page has Team Contribution, which explains what I think can be done here by each pair.