...
Pilot User Study
Conclusion
1. Title (0.
...
25 page)
Content:
...
Add the report title.
...
Write a concise and descriptive title for your report.
Below the title, list your:
Group number
Student names
Emails
Student numbers
Notes:
Make the title eye-catching and informative to immediately communicate the essence of your project
...
2. Introduction (
...
0.75 page)
Content:
...
Introduction to the
...
Project:
...
Define a conversational recipe recommendation agent
...
and its purpose.
Introduce Task-Oriented Spoken Dialogue Systems
...
Summarize Preliminaries: Describe what definitions and knowledge you utilized to complete this project.
...
State the goals your team aimed to achieve.
Tips:
...
Ensure clarity and conciseness. This section sets the stage for the reader.
...
(TOSDS), emphasizing their role in handling structured tasks like recipe recommendations.
Preliminaries:
Explain any key definitions, methodologies, or prior knowledge (e.g., intents, slots, NLU pipelines, or ontology design) that you used as foundational elements.
Goals:
State your team’s objectives clearly. Examples:
Build an agent capable of personalized recipe recommendations.
Etc.
Tips:
Set a positive tone for the report and provide context for why conversational agents are valuable in recipe recommendations. Mention briefly the importance of personalization (e.g., excluding allergens, adapting to dietary preferences).
...
3. Your Pipeline: How Does Your Conversational Agent Work? (2 pages)
...
Content:
...
Describe your pipeline, and give an overview.
...
Explain the overall functionality of your agent.
What problems does it solve?
What can users achieve by interacting with the agent?
:
Pipeline Overview:
Provide a high-level description of the architecture, from user input to recipe output.
Mention the key components seen in the diagram above as described in [TBC]Preliminaries and Quiz Materials.
Functionality:
Highlight the agent’s primary use cases.
Conversational Flow:
Walk through a typical user interaction.
Explain how user queries are processed through intent recognition, slot filling, and database queries.
Tips:
Use diagrams or flowcharts to visually illustrate the pipeline. Focus on making it understandable to both technical and non-technical audiences.
...
4. Intent and Slot Classifier (2 pages)
Content:
Role of Intent and Slot Classifier:
Explain the
...
Summarize training and testing
Performance analysis:
...
importance of these components in identifying user intentions and extracting relevant information (e.g., cuisine type, dietary restrictions).
Training and Testing:
Describe the datasets used for training/testing.
Highlight any preprocessing techniques or augmentation strategies employed.
Performance Analysis:
Present metrics:
Accuracy, precision, recall, F1 score
...
.
Use tables or confusion matrices to compare results across iterations.
Discuss challenges faced (e.g., ambiguous intents, overlapping slots).
Extensions to the
...
Model:
Mention improvements like:
Use of
...
pre-trained models
...
Tips:
...
Include visual aids such as tables or charts to present performance data effectively.
...
(e.g., BERT or GPT-based embeddings).
Hyperparameter tuning and architectural modifications.
...
5. Exclusion (2 pages)
Content:
Implementation:
How exclusion works (e.g., filtering excluding ingredients, cuisines, and tagsmealTypes).
Approaches used: intent-based, slot-based, rule-based, classifier-basedDescribe the approach your team used to implement Exclusion into your model.
Tools/technologies: Integration of MARBEL, Prolog, Python, and ontology updates.
Pros and Cons:
Discuss the strengths and limitations of the exclusion mechanism you choose. What can your exclusion do, what can it not do?
Testing:
Comparison of inclusion-only vs. exclusion models.Examples of exclusion in action (e.g., excluding dairy, gluten, or specific cuisines).
Performance Analysis:
Accuracy with and without exclusion.
Trade-offs and impact on user satisfaction.
Pros and Cons:
Discuss strengths and limitations of the exclusion mechanism.
Tips:
Use examples and data to illustrate the effectiveness of the exclusion approach.
Be critical and discuss what could be improved.
...
Describe additional enhancements:
New functionalities, filters, capabilities added.
Improvements in user interaction and experience (e.g., better response generation, conversational adaptability).
Explain the motivation and impact of these extensions.
...
Highlight how these extensions make the bot stand out beyond baseline requirements.
...
7. Pilot User Study (1
...
page)
Content:
Setup:
Who were the users?
What tasks were they asked to perform?
Methodology for data collection (e.g., surveys, observation, interaction logs).
Results:
Quantitative data: Success rates, error rates, average response time, etc.
Qualitative data: User feedback, observations of user behavior.
Analysis:
Key findings: What worked well? What needs improvement?
Lessons learned and implications for future work.
...