Course overview for the Project Conversational Agents
...
This page contains all pertinent project information. Go through it section by section to find out how to succeed in this course and find any relevant details you need to successfully complete this project.
...
Project Introduction
This project is aimed at learning how to build focuses on building a task-based conversational agent that can talk with and understand what a user says in reply, using speech as the main modality of interaction. In this project, your agent is tasked with capable of recommending recipes from a given recipe database to a user. Users should be able to provide input on their preferences and other constraints that a recipe they would like should satisfy. To facilitate communication with a user, the agent should not only talk but should also show relevant information about the recommendation task to a user. In other words, it should offer a spoken as well as a visual interface, and the agent needs to make sure there is alignment between these two interaction modalities.
One of the main challenges of designing such an agent is to make it work for all kinds of different users. The recommendation task itself may seem straightforward enough, but making your agent able to reply to whatever a user might say is not so easy. That is because users always vary in the way they approach a task, what they say, and how they say it, and conversational interaction in principle allows for all of this variation. First, there are many different ways in which a conversation may be conducted and many different ways in which a user can phrase what they want from the agent. A user can specify different aspects of a recipe that the recipe it will finally select should satisfy (e.g., type of ingredients, cooking duration, type of course, etc.). Second, the recipe recommendation domain already is a broad knowledge space that the agent needs to be able to handle to understand what the user is looking for. The agent will have to reason over its database of recipes to filter for recipes that fit the user’s preferences.
At the start of the project, you are provided with a Prolog database of close to 1,000 recipes and an agent template to get started. You will need to develop basic capabilities for your conversational agent, each of which will provide the agent with a basic understanding of what a user says, of how to organize the conversation, and what to show when to a user during the interaction. While developing your agent, it will be important to continuously and systematically test it and improve it based on your findings. This should result in a working agent, but it will still be pretty basic…
...
You are challenged to extend your agent with your own flare and ideas (with our assistance), which will enable you to earn a higher grade. At the end of this project, you and your team should have a fully functioning conversational agent that can assist users by recommending a recipe they would like to cook!
...
database of nearly 1,000 options. The agent will use speech as its primary mode of interaction while also incorporating a visual interface to display relevant information, ensuring alignment between these two modalities. Users will interact with the agent by specifying preferences and constraints, such as ingredients, cooking time, or type of dish, and the agent must understand and respond effectively to their inputs. A key challenge is designing an agent that can handle the variability in how users express their needs and conduct conversations, while also reasoning over the broad knowledge space of recipes to provide accurate recommendations. Starting with a Prolog database and a basic agent template, the project will involve developing essential conversational capabilities, systematically testing the agent, and refining its performance. Teams are encouraged to add creative features to enhance the agent’s functionality, culminating in a fully operational system that can assist users in finding recipes they would love to cook.
...
How is this Project Course Organized?
...