Introduction to the NLU Pipeline Project

Welcome to the Natural Language Understanding (NLU) Pipeline Project! This comprehensive system is designed to enable interaction with a conversational agent, leveraging cutting-edge technologies for dialogue management, natural language understanding, and recipe recommendations. Whether you are exploring machine learning, dialogue systems, or speech processing, this project offers a hands-on opportunity to build and integrate a sophisticated conversational AI pipeline.

...

Pipeline Overview

Core Components

1. Dialogflow (Initial Step):

   • Purpose: Provides a pre-built NLU solution to kickstart integration with the MARBEL agent.

   • Use Case: Allows the MARBEL agent to handle user interactions before the intent and slot classifier and WHISPER integration are complete.

   • Why Dialogflow?: By using Dialogflow, you can test and update the MARBEL agent’s dialogue management without waiting for the intent-slot classifier and ASR/TTS modules.

2. Intent and Slot Classifier (Custom NLU):

   • Purpose: Replaces Dialogflow to provide a self-built, fine-tuned classifier for interpreting user input.

   • Technology: BERT-based transformer model tailored for intent classification and slot filling.

   • Example:

     • Input: "Find me a Japanese recipe for lunch."

     • Output: Intent: addFilter, Slots: {cuisine: Japanese, mealType: lunch}

3. MARBEL Agent:

   • Purpose: Manages dialogue flow and user interaction through structured conversational patterns.

   • Capabilities:

     • Handles recipe selection, filtering, and confirmation tasks.

     • Manages unexpected user inputs and integrates with the recipe database.

4. Prolog Recipe Database:

   • Purpose: Provides structured storage and retrieval for recipes.

   • Integration: Works with the MARBEL agent to recommend recipes based on user preferences.

5. WHISPER (ASR and TTS):

   • Purpose: Adds speech capabilities to the conversational agent.

   • Features:

     • Converts user speech to text (ASR).

     • Translates agent responses into speech (TTS).

...

Why Start with Dialogflow?

To ensure the MARBEL agent can be developed and tested in parallel with the NLU and ASR/TTS systems, this project starts with Dialogflow as the initial NLU backend. Dialogflow allows you to:

• Define intents and entities easily using a GUI.

• Create training phrases for various intents and test responses immediately.

• Focus on refining the MARBEL agent’s dialogue patterns without waiting for the custom NLU pipeline.

Transitioning from Dialogflow

Once the intent and slot classifier is operational, you will replace Dialogflow with the self-built classifier, gaining full control over NLU customization and performance optimization.

...

Project Goals

The project is designed to:

• Provide a seamless transition from a pre-built to a custom NLU system.

• Demonstrate the application of transformers for intent classification and slot filling.

• Offer hands-on experience in building and integrating NLU, dialogue management, and speech systems.

...

Pipeline Components in Detail

1. Dialogflow:

   • Used for initial intent and slot classification.

   • Allows rapid prototyping and testing of MARBEL agent functionalities.

2. Intent and Slot Classifier:

   • Replaces Dialogflow once operational.

   • Provides a robust, customizable, transformer-based solution.

3. MARBEL Agent:

   • Designed for structured, multi-turn dialogue interactions.

   • Handles recipe selection, filtering, and confirmation tasks.

4. Prolog Recipe Database:

   • Offers efficient recipe retrieval based on user preferences.

5. WHISPER:

   • Adds ASR and TTS functionalities for speech-based interaction.

...

Development Workflow

The development workflow is divided into three structured phases to guide students through building and extending the NLU pipeline:

...