Welcome to this page with an overview of some of the required background information for this project. You most likely have already seen some of this information in other courses, but some of it will also be new to you. We expect each of you to have a decent understanding of the material listed below, as that is necessary to be able to complete this project successfully. You need to complete a quiz, where the material presented below can be used to prepare.
Page Overview
| Table of Contents | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Git and GitHub
We use GitHub Classroom to provide you with the initial agent code. GitHub is acode hosting platform for version control and collaboration. You need to join the GitHub classroom and use it for developing and sharing your code, and for storing and updating all the deliverables in this project. In order to help you understand how to do that, we will introduce you to some basic readings and a tutorial to help you gain knowledge of how to use git. Git is a common tool used by many coding teams worldwide to develop code in tandem and facilitate its alignment. Getting to know git as part of this course will surely be of benefit to you in the long term.
Git
Git is a tool used by developers to manage and track changes in their code or files. Think of it like a magical filing cabinet that remembers every version of a document or project you’ve ever worked on.
Imagine you’re writing a book. You make changes, but then decide you liked the way it was two days ago. Git lets you go back and see what it looked like back then.
It tracks what changes were made, who made them, and when.
It’s designed to help teams work together without accidentally overwriting each other’s work. It merges everyone’s contributions intelligently.
While Git works locally on your computer, it also pairs with tools like GitHub to store a backup of your work online.
Key Features:
Version Control: Keeps track of all the changes to a file or project.
Branching: You can create “branches” to work on different features or ideas without messing up the main version.
Undo Mistakes: If something breaks, you can roll back to a previous version.
GitHub
GitHub is like an online home for Git projects. If Git is your magical filing cabinet, GitHub is the cloud storage where you can share that cabinet with others.
It’s a website where people can store, share, and back up their Git projects online.
It ensures your work is safe even if something happens to your local files (like your computer crashing).
It makes it easy for teams to work together because everyone can see the latest version of the project and contribute their own changes.
GitHub also adds tools for collaboration like:
Issues: A way to track bugs or tasks.
Pull Requests: When someone suggests a change, the team can review it and decide whether to include it.
Actions: Automate tasks like testing your code every time there’s a change.
Git Commands
| Info |
|---|
You can do just the basics reading, just the interactive tutorial, or both. There is also a more in-depth explanation of each command on the third page. |
...
If you want to know more (not required): Everything on Git.
Git Merging and Conflicts
https://www.simplilearn.com/tutorials/git-tutorial/merge-conflicts-in-git
Git Best Practices
| Info |
|---|
One of the most important takeaways from the link below is that: Commits are Supposed to Be Small and Frequent Whenever you have made a single logical alteration, you can commit code. Frequent commit helps you to write brief commit messages that are short yet informative. Also, it will provide significant meaning for those who may be reading through your code. |
...
| Note |
|---|
In the case of this project commits must be made with certain specifications and things in mind please read Commits for more information. |
...
Machine Learning Basics Relevant to This Course
Note: We do not expect you to master all aspects of machine learning. Instead, focus on the following fundamental concepts that are directly related to this course.
What is Machine Learning?
Machine learning (ML)is a type of Artificial Intelligence (AI) that allows computers to learn and make decisions without being explicitly programmed. It involves feeding data into algorithms that can then identify patterns and make predictions on new data. In short, machines observe a pattern from data and attempt to imitate it in some way that can be either direct or indirect.
...
Using the pixels of an image to detect the presence or absence of a cat (image pixels → cat or no cat)
Using the movies you've liked to predict other movies you may enjoy (liked movies → recommended movies)
Using someone's words to predict whether they're happy or sad (text input → happy or sad)
Using a raw audio file to predict a transcript of the audio (audio file → transcribed text)
Why do we need Machine Learning?
Machine learning is able to learn, train from data, and solve/predict complex solutions which cannot be done with traditional programming. It enables us with better decision making ands solve complex business problems in optimized time. Recent advancements in AI have been propelled by machine learning, particularly its subset, deep learning. Additionally, compared to black-box agents like Dialogflow, developing our own machine-learning models provides greater control, enabling continuous improvement over time.
How do we train and evaluate a model?
When developing a machine learning model, one of the fundamental steps is to split the data into different subsets: training, testing, and validation datasets.
...
Training means building the model by learning patterns and parameters from the training dataset.
Testing involves assessing the model’s performance using the test dataset.
Hyperparameter tuning involves adjusting the model's hyperparameters to optimize its performance.
Model inference refers to using the trained model to make predictions or draw conclusions from new, unseen data. This step leverages the model’s learned patterns to apply it to real-world situations or new inputs.
What is Classification in Machine Learning?
Classification is a supervised machine learning method where the model aims to predict the correct label or category for a given input. In classification, the model is trained using labeled training data, learns to identify patterns, and is then evaluated on test data to assess its performance. Once trained and evaluated, the model can be used to make predictions on new, unseen data.
For example, a classification algorithm (classifier) might learn to predict whether a given email is spam or not, as illustrated below. This is a binary classification task, where the goal is to categorize the input data into two mutually exclusive classes. In this case, the training data is labeled with binary categories, such as "spam" and "not spam," "true" and "false," or "positive" and "negative." These labels guide the model in learning the differences between the two categories, allowing it to make accurate predictions when exposed to new data.
...
What is a deep neural network and how does it work?
A deep neural network (DNN) is a machine learning model, that makes decisions in a manner similar to the human brain, by using processes that mimic the way biological neurons work together to identify phenomena, weigh options, and arrive at conclusions.
...
Optimizing the loss function: During training, the goal is to find the “best” values of the model parameters (weights and bias) that minimize the loss function based on the training dataset. This process is known as optimization.
...
A General Pipeline of Task-Oriented Spoken Dialogue Systems
Spoken dialogue systems (SDSs) have been the most prominent component in today’s virtual personal assistants, such as Microsoft’s Cortana, Apple’s Siri, Amazon Alexa, Google Assistant, and Facebook’s M. Unlike chitchat, task-oriented SDSs aim to assist users with a specific goal, for example, recommend a recipe or booking a hotel.
A classical pipeline architecture of a task-oriented spoken dialogue system includes key components:
...
This hands-on approach will provide insight into developing and refining key elements of a dialogue system.
ASR with WHISPER
Automatic Speech Recognition (ASR) is a key component in the pipeline architecture, which converts spoken language into text. It enables machines to interpret and transcribe human speech, allowing for seamless interaction between users and applications through voice commands.
Whisper is a commonly used general-purpose speech recognition model developed by OpenAI. It is trained on a large dataset of diverse audio and is also a multi-tasking model that can perform multi-lingual speech recognition, speech translation, and language identification.
NLU with BERT
Unlike chitchat, task-oriented dialogue’s pattern is restricted by a dialogue ontology, which defines all possible intents, slots, and their corresponding candidate values in specific domains. The NLU component maps a user’s utterance to a structured semantic representation, which includes the intent behind the utterance and a set of key-value pairs known as slots and values. This mapping enables dialogue systems to understand user needs and respond appropriately. For example, given the transcribed utterance “I want to cook Italian pizza“, the NLU model can identify: the intent as “addFilter“ and the value of the slot “ingredienttype“as “italian pizza“.
...
| Info |
|---|
You can do just the basic reading of the above. There are also more in-depth explanations on the following third page. |
...
LLMs on Hugging Face
Hugging Face is an AI community and platform that offers an easy-to-use interface for accessing and utilizing pretrained large language models (LLMs) like BERT released by various organizations and researchers. Here is a simple example of how to use this model to get the features of a given text in PyTorch:
...
You can easily switch to different models by changing the model name, e.g., bert-large-uncased, which has 340 million parameters. Generally, models with more parameters tend to deliver better performance.
...
HTML, Javascript, and Bootstrap
You will be developing a few basic web pages to provide some visual support to a user while they is conversing with your agent. We assume you are familiar with basic forms of HTML, the markup language for developing a webpage. If not, please check out this https://www.w3schools.com/html/default.asp to get you started.
...
To gain an understanding of Bootstrap, this https://www.w3schools.com/bootstrap4/default.asp will be very useful. To familiarize yourself with some of the basic components of Bootstrap, take a look at the first few items in the tutorial. We recommend you read at least up to the item on https://www.w3schools.com/bootstrap4/bootstrap_buttons.asp. The Tutorial will be useful for later reference to look up how you can change colors and use, for example, a progress bar.
...
Prolog
You will develop your recipe recommendation agent using MARBEL and SWI Prolog. The MARBEL agent implements a dialog management engine that you will use. You do not need to change this agent. You are, however, allowed to modify it if you like. The focus will be mostly on using Prolog to provide the agent with the knowledge it needs and to make it smarter by providing it with some logic related to its recipe recommendation task.
...