...
In this project, you will be developing a conversational cooking assistant that uses speech to interact and is able to conduct a conversation for selecting a recipe to cook. Automatic Speech Recognition (ASR) has matured to a level where it is possible to categorize a user's utterances adequately to make sense out of them. Also, Speech Synthesis (TTS) can be used to produce well-pronounced spoken utterances from written text. Yet, conversational agents have not become mainstream, and whoever has used a home assistant (Google Home or Apple Siri) has experienced being misunderstood. These assistants are typically able to perform well on basic Question-Answering (QA) interactions, which most of the time consist of just two conversational turns: a question and an answer. However, conducting longer conversations tends to be more challenging. This is because a longer conversation can take (too) many directions and the chance that a user says something unexpected significantly increases. We will investigate this challenge in this project.
Our conversational agent is In this project, you will develop a cooking assistant that should be able to support a user in the food domain. There are tons of recipes on the internet and sometimes finding something to choose from can prove to be difficult, especially when you want something specific. Sometimes at night, I think to myself “I really want a Mexican recipe, that has less than 10 steps and no bell peppers in it”, but how could I even go about finding my perfect recipe? Maybe with a super huge database of recipes and a bot! In this project, you will be exploring a solution for this problem and develop a cooking assistant that conversationally assists a user in selecting a recipe based on a variety of filters. We shall provide a bundle of files that still need a bit of work done to complete themconversationally assists a user in selecting a recipe based on a variety of filters, whereas the instruction of the recipe itself is not in the scope of the Project MAS course. We chose to focus on the recipe selection activity of cooking support, since several challenging elements of building a conversational agent come together in this segment. First, there are many different ways in which this conversation may conducted, with different aspects of recipes that may be specified by the user to come to a selection (type of ingredients, cooking duration, course, etc.), and the option to mention multiple of those elements in one utterance. Second, the agent has to cover a broad knowledge space to understand what the user is looking for. Third, the agent has to reason over its database of recipes to filter for recipes that fits the user’s preferences.
You are provided with a prolog knowledge base of close to 1,000 recipes and their components, that still requires an effort of your group to make it usable by the cooking assistant. In the instructions we give you we walk you through how to fill in the blanks. If you fill in the blanks the agent should work… but it will be pretty basic…
...
Team Forming and Initial Set Up
By At the end start of the weekcourse, you will need to form are expected to have formed teams of 6and begin to prepare for your project. Roles are divided in this project into three categories, which is divided into three roles: Dialogflow and Filters (2 team members); Visual Support (2 team members); and Patterns and Responses (2 team members). The number in parenthesis is the number of people we recommend you assign to each category. You will still need to work together in many regards, so make sure there is still communication between categories. Together you will set up your project and familiarize yourself with the project toolstwo team members with a similar role will work together, but the three teams of two need to communicate well to make sure those three aspects of the agent are integrated.
Instructions for the tools and setup can be found here:
...