Content Comparison

This tutorial will show shows you how to transcribe the audio from a file on your computer using dialogflowDialogflow. Dialogflow was made to be used for conversations, but as it sends a transcription of what was said we can use it to transcribe audio as well.

You should now have the following set up at the end:

1. SIC is installed on your laptop

2. Redis is running on your laptop

...

In addition to the given preliminaries, you will also need to have PyAudio installed on your virtual environment.

Approach

This tutorial will show you how to convert audio to text. We’ll split this up into a couple parts

1. Converting your an audio file to a .wav file

2. Starting the dialogflow Dialogflow component

3. Transcribing the audio file

Converting to

...

.wav format

To be able to read the audio in python, its easiest to convert it to a .wav file. Depending on which file type you have this might need to be done differently, but here is an example using ffmpeg. Make sure to convert it to mono 16bit PCM little-endian audio (this is what pcm_s16le means).

ffmpeg -i my_audio.mp3 -codec:a pcm_s16le -ac 1 -ar 44100 my_audio.wav

Installing and starting

...

Dialogflow

To start dialogflow, you will likely need to install additional packages. You can do this with

pip install -r sic_framework/services/dialogflow/requirements.txt

Now that we have everything for dialogflow installed, we can start the component.

cd sic_framework/services/dialogflow
python3 dialogflow_service.py

...

1. First, start the SIC Dialogflow service, you should see something like this:

(base) user@laptop:~/framework/sic_framework/services/dialogflow$ python3 dialogflow_service.py 
[SICComponentManager 192.168.0.181]: INFO: Manager on device 192.168.0.181 starting
[SICComponentManager 192.168.0.181]: INFO: Starting component manager on ip "192.168.0.181" with components:
[SICComponentManager 192.168.0.181]: INFO:  - DialogflowService

...

...

...

If you have trouble installing dialogflow locally, you can also try to start the component using docker. Make sure redis is not running anywhere else, and in the framework folder use

docker compose up dialogflow

Getting a key

If you don’t already have a key, check out Getting a google dialogflow key

If everything went right, you should have have a your_dialogflow_key.json

Create a Dialogflow agent

Follow the following steps to create a dummy Dialogflow agent:

1. Go to https://dialogflow.cloud.google.com/#/getStarted and click “Create agent” in the top left

2. Select the right Google project (the one you created during the dialog flow key setup)

3. Click on create in the top right

Transcribing the audio

Alright! Now that we have everything set up we can start transcribing the audio.

...

1. The dialogflow component is running

2. You have a dialogflow key

3. A .wav audio file in the folder you are working in

In a new python file (or check out TODO) copy the following code:

import jsonthreading
import threadingpyaudio
import wave

import pyaudiojson

from sic_framework.core.message_python2 import AudioMessage
from sic_framework.services.dialogflow.dialogflow_service import DialogflowConf, GetIntentRequest, Dialogflow, \
    StopListeningMessage, QueryResult, RecognitionResult

...

Now we get to more interesting stuff. The dialogflow Dialogflow component will send back a lot of information, so we will have to handle that, and extract the transcription.

First, we’ll create an event. We’ll set this event whenever dialogflow Dialogflow has detected the end of a sentence. That way we can ask dialogflow Dialogflow to listen to the next immediately after. Its easiest to use a threading.Event, because dialogflow Dialogflow will signal the end of a sentence at an arbitrary point.

...

We are still working on how to properly end a conversation after it is done, but google’s Google’s documentation has little mention of this. If you find it, let us know!

...