...
Evaluation is an essential step to evolve your cooking assistant recipe recommendation agent from an agent that can conduct a basic conversation , to an agent that many users can have a conversation with. Robustness is the keyword here. that offers minimal functionality to a user to an agent that can provide a great user experience for all users. Effectiveness, efficiency, robustness, and user satisfaction are important metrics here. An agent is effective if it can guarantee task completion. Such an agent adequately assists a user with completing the task at hand. It is efficient if the number of turns needed to achieve the goal in a conversation is optimized. It is robust if it can handle all phrases that users might reasonably be expected to say and can handle the different directions that a conversation may take. Finally, a conversational agent is satisfying if a user experiences the conversation as satisfying.
By testing, you can find out the deficiencies of the agent that need to be addressed so that it will not break down easily. This is typically done in a cyclical manneran agile manner, starting with a simple agent that functions minimally, where rounds of tests and improvements follow one another. You will follow two stages in the current project: system testing and a user study, which we will explain in further detail in the following subsections. The two stages will be reported on by your group in the end report.
First, you will focus on agent testing:
...
Second, at the end of the project, you will do a user study:
...
You will need to document agent testing and the user study that your team performed in your final report.