Your Intent and Slot Classifier

1 Getting Started
- 1.1 Purpose of the Project
- 1.2 Repository Overview
- 1.3 Explanation of Key Modules
  - 1.3.1 run_train_test.py
  - 1.3.2 dataset.py
  - 1.3.3 model.py
  - 1.3.4 train.py
  - 1.3.5 evaluation.py
  - 1.3.6 predict.py
  - 1.3.7 utils.py
- 1.4 Arguments
- 1.5 Use
  - 1.5.1 2. Training the Model
  - 1.5.2 3. Evaluating the Model
  - 1.5.3 4. Running Inference
  - 1.5.4 5. Run with ASR

Getting Started

Natural Language Understanding (NLU) is a core component of conversational AI systems, enabling machines to interpret and act on user input in natural language. This Intent and Slot Classifier project is designed to help students understand the pipeline involved in building an NLU model that performs intent classification and slot filling. These tasks allow AI models to classify a user's goal (intent) and extract key information (slots) from their input.

For instance, given the input:
"Find me a Japanese recipe for lunch,"
the model would classify the intent as "find_recipe" and extract slots like "cuisine: Japanese" and "mealType: lunch".

This project uses BERT, a state-of-the-art transformer-based language model, to perform these tasks effectively. The system provides end-to-end functionality for training, evaluating, and running inference on the classifier.

Purpose of the Project

The primary goals of this project are:

To demonstrate the application of transformers for intent classification and slot filling.
To provide students with hands-on experience in creating and working with NLU pipelines.
To showcase best practices for preparing datasets, building models, and evaluating their performance.

To complete these goals you must go through all subpages in this section starting with Ontology.

Repository Overview

Find the Intent and Slot Classifier files under /pca-agent-2025/social-interaction-cloud/sic_framework/services/nlu/utils

The repository is structured as follows:
utils/
├── checkpoints/          # Directory for saving trained model checkpoints
├── data/                 # Directory containing data files (ontology, train/test datasets)
│   ├── ontology.json     # Ontology file containing intents, slots, and synonyms
│   ├── train.json        # Training dataset
│   ├── test.json         # Test dataset
│   └── synonyms.json     # Synonyms for slot normalization
├── data_processing.py    # Utilities for additional data preprocessing (if needed)
├── dataset.py            # Dataset preparation and preprocessing module
├── evaluation.py         # Model evaluation and metrics generation
├── run_train_test.py     # Main script to run training, evaluation, and inference
├── model.py              # Defines the BERT-based model architecture
├── predict.py            # Inference module for predicting intents and slots
├── requirements.txt      # Python dependencies for the project
├── train.py              # Training module for the intent-slot classifier
└── utils.py              # Helper functions for argument parsing, slot normalization, and synonym resolution

Explanation of Key Modules

`run_train_test.py`

This is the central script for orchestrating the intent and slot classifier independently. It integrates data preparation, training, evaluation, and inference, all controlled via command-line arguments.

`dataset.py`

This module handles data loading, preprocessing, and BIO tagging. It uses the NLURecipeDataset class for creating PyTorch-compatible datasets.

`model.py`

Defines the BERTNLUModel, a BERT-based architecture with separate heads for intent classification and slot filling.

`train.py`

Implements the training routine, including optimization and backpropagation.

`evaluation.py`

Provides functionality to evaluate the model’s performance, generating accuracy scores and classification reports.

`predict.py`

Handles inference by predicting intents and slot tags for new inputs. It includes functions for extracting and normalizing slots using the ontology.

`utils.py`

Contains helper functions for:

Parsing command-line arguments.
Loading and normalizing synonyms.
Extracting and formatting slot values from predictions.

Arguments

The utils.py module defines the command-line arguments. Here’s a summary:

Argument	Type	Default	Description

Argument	Type	Default	Description
`--ontology_path`	`str`	`./data/ontology.json`	Path to the ontology JSON file.
`--train_data`	`str`	`./data/train.json`	Path to the training dataset.
`--test_data`	`str`	`./data/test.json`	Path to the test dataset.
`--model_save_path`	`str`	`checkpoints/model_checkpoint.pt`	Path to save/load the trained model weights.
`--train_model`	`bool`	`False`	Train the model when this flag is set.
`--evaluate`	`bool`	`False`	Evaluate the model on the test dataset when this flag is set.
`--num_epochs`	`int`	`2`	Number of epochs for training.
`--batch_size`	`int`	`16`	Batch size for training.
`--learning_rate`	`float`	`5e-5`	Learning rate for the optimizer.
`--max_length`	`int`	`16`	Maximum sequence length for tokenization.
`--seed`	`int`	`42`	Random seed for reproducibility.
`--inference_text`	`str`	`None`	Text input for running inference.
`--show_dist`	`bool`	`False`	Show the intent and slot distribution in the dataset.

Use

Due to how the imports are set up and because the intent and slot classifier is part of the social-interaction-cloud python package when you make changes you need to make sure to reinstall the social interaction cloud via pip install .

1. Viewing Dataset Distribution

To analyze the distribution of intents and slots in the dataset:

python run_train_test.py --show_dist

2. Training the Model

To train the model using the training dataset:

python run_train_test.py --train_model

3. Evaluating the Model

To evaluate a pre-trained model on the test dataset:

4. Running Inference

To predict the intent and slots for a given input text:

5. Run with ASR

When complete with this section read Run your Conversational Agent | Run your Intent and Slot Classifier with WHISPER to connect your intent and slot classifier with WHISPER

Project Conversational Agents 2025