...
Setting up your user study: Your team will run the pilot user study for your own agent and collect what you think is relevant data to analyze the performance of your agent. Consider some of the metrics that were already explained in on the Agent Testing sectionand Pilot User Study page: effectiveness, efficiency, robustness, and user satisfaction. So think about what you want to evaluate and how to do that. Also, consider that the greatest advantage of having another team test your agent is that they likely will interact differently and may have different conversations with your agent than you have seen before in your own agent testing (that’s why it makes sense to already involve people from outside your team during agent testing).
...