If you ever baked a cake or cooked a meal from a recipe in a cooking book or on a website opened in your phone/tablet, you probably recognize the hassle of repeatedly checking the details of a step and having to scroll through a screen with hands dirty from cooking. Throughout the Project MAS course, you will be working on the solution to those sorrows: a cooking assistant that conversationally guides the user through a recipe and responds to his/her questions.

Automatic Speech Recognition and Speech Synthesis have come to a proper level for a user's utterances to be (mostly) understood and for the agent to clearly articulate utterances from written text. Yet, conversational agents have not become mainstream, and whoever has used a home asssistant (Google Home or Apple Siri) has known feelings of being misunderstood and ultimately irritation. Especially conversations tend to be challenging, as the conversation can take too many directions and the chance that the user says something unexpected increases. In this assignment you will be developing an agent that is able to conduct an extended recipe instruction conversation, and try to prevent or solve such feelings of misunderstanding of the user as the conversation progresses.

The project MAS course builds upon the basics taught in the Multi-Agent Systems course. This means that you will develop the agent using GOAL and Prolog as knowledge representation language. While the GOAL assignments in the MAS course were linked to the BW4T environment, this course will make use of the Social Interaction Cloud (SIC) environment. In addition, Google Dialogflow is used to handle the speech of the agent, as well as the interpretation of user utterances. The first objective in this course is therefore to set-up and connect those modules to the agent. In the course, you will be working in groups of six, going through the different stages of chatbot development: collecting content, designing conversations, implementing conversation patterns, adding supporting visuals, testing the agent, extending on the content/patterns, and further testing the agent. This will in the end result in the agent program, as well as a report in which you describe its different components and how it was tested. This document will provide a week-by-week description of the targets, while you have the freedom to implement certain components according to your own insights and extend the agent beyond the basic requirements as far as you like.

Assignment Objectives and Instructions

Once you know how to get started, your can start to design and develop a prototype of a Cooking Assistant. This assignment requires good communication and contribution throughout the assignment. You will work in a group of six students on three interconnected components of the cooking assistant. It makes most sense to divide your group in three pairs who will work separately on these components. The pair working on Recipe Selection (RS) will mainly work on selecting the recipe, while Recipe Instruction (RI) will work on instructions related to the conversational agent design. The Visual Support (VS) pair will focus on adding items on a display (e.g., text, images, lists of items, buttons) to support the interaction flow developed by the other two pairs. Although the RS and RI pairs can design their solution relatively independently from each other, team work is still very much required between the pairs to ensure consistency between the visuals and the interaction design of the RS and RI pairs. Below is a weekly distribution of the MAS project assignment. Click on the week number to get more details related to the tasks you are advised to complete in that week.