Project Introduction and Overview

Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) have matured to a level where it is possible to translate a user's utterances into text (ASR) and to classify text into intents (NLU) to make sense of what a user says. Also, Text-To-Speech (TTS) can be used for speech synthesis to produce well-pronounced spoken utterances from written text. Yet, conversational agents have not become mainstream, and whoever has used a home assistant (Google Home or Apple Siri) has experienced being misunderstood. These assistants are typically able to perform well on basic Question-Answering (QA) interactions, which most of the time consist of just two conversational turns: a question and an answer. However, conducting longer conversations tends to be more challenging. This is because a longer conversation can take (too) many directions, and the chance that a user says something unexpected significantly increases. We will investigate this challenge in this project.

In this project, you will be developing a conversational recipe recommendation agent that uses speech to interact and is able to conduct a conversation for selecting a recipe to cook. Your agent should be able to assist a user in selecting a recipe using a variety of filters. The agent does not need to be able to assist a user with the instruction steps of the recipe itself, which is out of scope of the Project MAS course. We chose to focus on the recipe selection activity since it already poses several challenges for building an effective and robust conversational agent. First, there are many different ways in which this conversation may be conducted, and many different ways in which a user can phrase what it wants from the agent. A user can specify different aspects of a recipe that the recipe it will finally select should satisfy (e.g., type of ingredients, cooking duration, type of course, etc.). Second, the recipe recommendation domain already is a broad knowledge space that the agent needs to be able to handle to understand what the user is looking for. The agent will have to reason over its database of recipes to filter for recipes that fit the user’s preferences.

You are provided with a Prolog knowledge base of close to a 1,000 recipes and their components, that still requires an effort of your group to make it usable by the recipe recommendation agent. In the instructions, we walk you through the procedure by pinpointing aspects of the agent that need to be altered or filled in. If you fill in these blanks, the agent should work, but it will still be pretty basic…

The project does not stop there. We challenge you to extend this basic bot with your own flare and ideas, so as to earn a higher grade. Read a bit more about this in our Extensions section at the end of your project.

During development and after finishing the agent, it is essential to do some testing! This will also involve having another team conversing with your agent, and checking out another team’s agent yourself. You can read about this in our System testing and User study section.

Apart from a working and tested agent, your project work will result in an end report in which you describe its different features and its performance according to the testing, among others. More information on that can be found in the 2023: End Report Section.

For now, let’s get started by dividing work among teammates.