Open

This page contains all pertinent project information. Go through it section by section to find out how to succeed in this course and find any relevant details you need to successfully complete this project.

Project Introduction

This project is aimed at learning how to build a task-based conversational agent that can talk with and understand what a user says in reply, using speech as the main modality of interaction. In this project, your agent is tasked with recommending recipes from a given recipe database to a user. Users should be able to provide input on their preferences and other constraints that a recipe they would like should satisfy. To facilitate communication with a user, the agent should not only talk but should also show relevant information about the recommendation task to a user. In other words, it should offer a spoken as well as a visual interface, and the agent needs to make sure there is alignment between these two interaction modalities.

One of the main challenges of designing such an agent is to make it work for all kinds of different users. The recommendation task itself may seem straightforward enough, but making your agent able to reply to whatever a user might say is not so easy. That is because users always vary in the way they approach a task, what they say, and how they say it, and conversational interaction in principle allows for all of this variation. First, there are many different ways in which a conversation may be conducted and many different ways in which a user can phrase what they want from the agent. A user can specify different aspects of a recipe that the recipe it will finally select should satisfy (e.g., type of ingredients, cooking duration, type of course, etc.). Second, the recipe recommendation domain already is a broad knowledge space that the agent needs to be able to handle to understand what the user is looking for. The agent will have to reason over its database of recipes to filter for recipes that fit the user’s preferences.

At the start of the project, you are provided with a Prolog database of close to 1,000 recipes and an agent template to get started. You will need to develop 10 basic capabilities for your conversational agent, each of which will provide the agent with a basic understanding of what a user says, of how to organize the conversation, and what to show when to a user during the interaction. While developing your agent, it will be important to continuously and systematically test it and improve it based on your findings. This should result in a working agent, but it will still be pretty basic…

Open

Therefore, after implementing the first 10 agent capabilities, you are challenged to extend your agent with your own flare and ideas (with our assistance), which will enable you to earn a higher grade. At the end of this project, you and your team should have a fully functioning conversational agent that can assist users by recommending a recipe they would like to cook!

How is this Project Course Organized?

Course Schedule

The project course Conversational Agents (code XB_0101) runs in period 3 (Jan to Feb 2024). Lectures each week are scheduled on Tuesdays from 13:30-15:15 in WN-KC137. Practical sessions are scheduled each week on Tuesdays from 15:30-19:15 and Fridays from 9:00-12:15.

Go here for the detailed https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2215575562.

Communication & Contact policy

For communication throughout this project course, we will use Discord. Using the project’s Discord server, you can communicate about and work on the project assignment outside classroom hours, with your team members, fellow students, and teaching assistants. For each group, we will create a group channel. In your group channel, you can privately talk with the other students in your group and your TA (no one else will have access).

Your group’s Teaching Assistant is your main point of contact for any questions you have about the course and the assignment. Discord is also the way for you to communicate with your TA.

For questions about the project course itself and for the project assignment, we have created two forum channels for these two topics. Within each of these channels, you can post your questions and/or find previously given answers to questions asked by other students.

If your question is not replied to (by other students or any of the TAs), then contact your own TA. If you have any other issues to report, always first contact your group’s TA via Discord. If the question or issue cannot be resolved with your TA, ask your TA to take this up with us. Only if after that you still have not received any response within a reasonable time (within a week that means), send the teachers a mail.

Inclusivity Statement

We strive to offer students a safe and inclusive classroom environment. We welcome the perspective of students of all ethnicities, genders, spiritual beliefs, and sexual orientations, among others. If you feel discriminated based on your identity, please report it to the course staff.

If you have a disability and require accommodations, please let us know before the beginning of the course so that we can discuss your accommodations and needs to make the course as accessible to you as possible.

Academic integrity and code of conduct

Plagiarism is not tolerated for both reports and code. You are welcome to use any code you see fit for your project, as long as you appropriately cite your sources both in the report and in the code itself (as comments). Behavior violating the VU rules for academic integrity will not be tolerated.

Getting Started!

Form a group of 6

You will be working on the main project assignment together in a group of 6 students. We will assign a teaching assistant (TA) to each group. Your TA will also monitor whether each of you individually sufficiently contributes to the deliverables. Individual contribution and participation in the course are required to pass the course. Enroll your group members on Canvas in a group.

Project Background Knowledge

Before you begin this project, please make sure that you understand the relevant background knowledge that you will need throughout the project. The background knowledge you need concerns the use of the git version control system, Google’s Dialogflow to transcribe and understand spoken natural language, Prolog for managing the dialog and reasoning about recipes, and the basics of HTML and Bootstrap to create a visually appealing webpage. You can learn about this as a team, but every member of your team must at least have a basic understanding of all of these concepts. You will also need this background knowledge to complete a few (see also the https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2215575562). So, to get up to speed, check out thehttps://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2215706657!

Project Installation

With a basic understanding of the concepts and tools that you will use in this project, you now should proceed with installing all the needed software and tools on your device. Some will be re-used from the MAS course, but others are new. Go to https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2215706685 which has all the details related to installation and computer setup and complete all instructions on this page.

The Project Assignment

You and your team should now be ready to get started on developing your own conversational agent capable of recommending recipes from a given recipe database.

Building your Agent

The core of this project surrounds building a recipe recommendation agent as a team. There are three kinds of tasks that you will encounter:

We encourage you to work on these assignments as a team as much as possible. Each step in building your agent tasks you with developing a new “Agent Capability” and completing the tasks associated with each capability will provide your agent with the ability to do something new. All three task types must be completed using the instructions in each corresponding section in order to complete the overall task of providing your agent with a new capability. Also, everyone in your team must (to a reasonable extent) understand all three task types and the work done within them.

Throughout the project go here for instructions on https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2216001572.

Testing

Testing your conversational agent continuously is very important during the project. Each week you should collect, analyze, and submit data about how your agent is performing (check out the https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2215575562). Your analysis of that data will help to improve your agent. However, testing your own agent yourself is not good enough because developers are not the best testers of their own software.

To make your agent perform robustly with arbitrary users, you should also conduct at least some more structured tests with users to get insight into how robust your agent really is. You can already get started with this from early on in the project by involving other users from outside your team (e.g., your friends, family, or students not part of your team) and asking them to interact with your agent. You will find that if you do this you will learn a lot about what needs to be improved and how you can improve your agent. In any case, at the end of this project, you will need to conduct pilot testing sessions where you and another team shall be paired to test each others' agents. You need to report on the findings from these sessions in your final report. You can read more about how to prepare for these sessions in the https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2257616897 section.

How to pass the course, deliverables & grading

Requirements for successfully completing the project

To complete this course, you must have participated in and completed the following:

• Completed the https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2215706685 on your own device/laptop.

• Signed up to the project’s Discord server with your own name (and not a nickname).

• Signed up to the project’s GitHub classroom and actively contributed to your group’s repository.

• Participated in the mandatory lab sessions twice a week, as also listed on the https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2215575562.

• Kept your individual https://socialrobotics.atlassian.net/wiki/spaces/MAS23/pages/2157019221 up to date by updating and committing it to your GitHub repo at least once every week; your log book needs to show you actively participated in the course.

• Completed individually all the Canvas quizzes related to the materials provided to you.

• Submitted as a team all https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2257518603 before the project’s deadline.

Assessment and Grading

• Your coursework is only assessed, and you will only receive a grade if all the requirements listed above are met.

• Your coursework is assessed based on the quality of your team’s deliverables. Check out the https://socialrobotics.atlassian.net/wiki/spaces/PM2/pages/2257518593 for details.

• The work is graded with a mark from 0-100. There must be a minimum of 55 to pass the course.

• Individuals may be rewarded a higher or lower grade than the group based on their effort and their contribution to the teamwork. We will among other things use your log books for this purpose.