Page Comparison

...

Before creating the demo, we need it’s essential to understand that the design of the Webserver WebserverComponent goes hand in hand with the requirements of the demo, as the Webserver WebserverComponent is the component that handles any event events from the a web client. So, in For this specific tutorial, the Webserver needs to handle the WebserverComponent should handle a button click event and send live transcripts transcribed produced by Dialogflow service to the a connected client over a WebSocket connection.

So, this This tutorial can split be divided into two parts:

1. The design of html HTML and Webserver

2. Creating the html

Creating the flask server in Webserver

WebserverComponent

1. How to create a button and catch a click event on it?

2. How to render Dialogflow intermediate transcripts dynamically in a text box?

3. The demo program connects all the servers components (NaoqiMicrophoneNaoqiMicrophoneSensor(or local microphone DesktopMicrophone), WebserverComponent, Webserver DialogflowComponent, Dialogflowetc)

The design of html and Webserver

1. How to create a button and catch a click event on it?

In the demo_pepper_guess_number.html, we need to Add add a button and make it listen to the click event. Once the button is clicked, the isStreaming isClicked flag will be set to true, and it sends will send the flag to the Flask server via WebSocket.

// Flag to track if the game should start
var isStreamingisClicked = false; 

// Get Eventa listenerreference forto the startstartButton button clickelement
var startButton = document.getElementById('startButton');

// Event listener for the start button click
startButton.addEventListener('click', function() {
  isStreamingisClicked = true; // Set the flag to start streaming
  sendFlagToServer(isStreamingisClicked);
});

// Function to send flag to Flask server
function sendFlagToServer(flag) {
  socket.emit('streamingclicked_flag', flag);
}

On flask server server, we need to create

1. Creating the flask server in Webserver

...

the Flaser server (webserver_pepper_tablet.py), we need to register a handler to react to the "clicked_flag" event.

# register clicked_flag event handler
@self.socketio.on('clicked_flag')
def handle_flag(flag):
    if flag:
        self.output_message(ButtonClicked(button=flag))

If the flag is set to true, a SICMessage “ButtonClicked” will be sent to the SIC channel as an output of this component. This output will be received on a callback on the demo side (demo_pepper_guess_number.py) to determine whether the button has been clicked or not.

2. How to render Dialogflow intermediate transcripts dynamically in a text box?

In the html, first add a textarea element inside the body section.

<div>
  <textarea id="textbox" rows="4" cols="50"></textarea>
</div>

In Javascript, we create a function to update the content of the textbox .

// Function to update the textbox
function updateTextbox(text) {
  document.getElementById('textbox').value = text;
}

When a web client receives an “update_textbox” event emitted from the Flask server, the event handler is triggered. Then it calls the updateTextbox(text) function passing the received text as an argument.

    // Event handler for receiving updated text
    socket.on('update_textbox', function(text) {
      updateTextbox(text);
    });

On the Flaser server (webserver_pepper_tablet.py), we emit the “update_textbox” event along with the transcript transcribed from Dialogflow.

self.socketio.emit('update_textbox', self.transcript)

The demo program connects all the components

The demo program is located in tests/demo_webserver/demo_pepper_guess_number.py.

There are many SIC components we need to connect to in this demo.

Components running on a pepper:

1. NaoqiMicrophoneSensor

2. NaoqiTextToSpeechActuator

3. NaoqiTabletComponent

Components running on a local machine:

1. WebserverComponent

2. DialogflowComponent

Here is the schematic overview of how this program works: The demo_pepper_guess_number progam serves as the user application control, sending an HtmlMessage as input for the WebserverComponent to render. Upon receiving the ButtonClicked message with a value of “True”, it sends a GetIntentRequest to the DialogflowComponent. The resulting RecognitionResult is received in the on_dialog callback, and upon obtaining the transcript result, a TranscriptMessage is sent to the WebserverComponent for dynamic transcript rendering in the above-mentioned textbox.

During each GetIntentRequest request, the resulting query_text generated by Dialogflow is passed to the extract_and_compare_number function. This function extracts a number from the query_text and compares it with the randomly generated number x. When the detected number is higher/lower than or equal to x, we will request the NaoqiTextToSpeechActuator to make Pepper say the corresponding response.

...