Page Comparison

This tutorial shows you how to run a simple pipeline (ASR + NLU) where Whisper transcribes your speech and feeds it into the NLU component to run an inference.

Steps:

Since the NLU component is not yet available on PyPI, for now, we will need to clone the repository and install it locally.

1. Clone the SIC repo

   1. git clone https://github.com/Social-AI-VU/social-interaction-cloud.git

2. Switch to the nlu_component branch:

cd social-interaction-cloud
git checkout nlu_component

3. Create and activate a virtual environment:

   1. If you are using pure Python environment

      Code Block
      python -m venv venv_sic source venv_sic/bin/activate

   2. If you are using Anaconda environment

      Code Block
      conda create -n venv_sic python=3.12 conda activate venv_sic

4. Install SIC, nlu and whisper dependencies from local repo: once changing the local code, you need this to compile all again.

   Code Block

...

3. pip install ."[whisper-speech-to-text,nlu]"

4. Start the Redis server + SIC components, correspondingly. (Don’t forget to run a redis server. See the details here https://socialrobotics.atlassian.net/wiki/spaces/CBSR/pages/2180415493/Getting+started#Step-1%3A-starting-Redis-on-your-laptop ):

   Code Block
   source activate venv_sic # Start the Redis server, followed by NLU and Whisper components redis-server conf/redis/redis.conf & run-whisper & run-nlu &

You can use the following command to manually stop the backend servers, correspondingly:

pkill -f "run-nlu"
pkill -f "run-whisper"
pkill -f "redis-server"

6. Open a new terminal and activate the same virtual environment you created earlier

7. Clone the sic_applications repo

   1. git clone https://github.com/Social-AI-VU/sic_applications.git

8. Add (trained) nlu model and ontology to the configuration folder sic_applications/conf/nlu. The default names are “model_checkpoint.pt" and "ontology.json".

   Image Modified

9. Run the demo

   Code Block
   cd sic_applications/demos/desktop source activate venv_sic python demo_desktop_asr_nlu.py