Wizard-of-Oz interface - Restructured

Introduction

The Wizard-of-Oz interface offers manual access to the capabilities of the robot. By getting manual access to the camera feed, microphone input, gestures, TTS and tablet, the user can partially or fully tailor the robot interaction manually. As such, if the interaction script doesn’t work as expected, the user can step in and offer human input.

Docker name: wizard

Input

sensors: see Pepper/Nao robots - restructured
actuators: see Pepper/Nao robots - restructured
service(s): stream_video, stream_audio, webserver
parameters (optional):
- JSON configuration file (for more details, refer to initialization)

Note: To launch the service itself, it does not require any other sensors, actuators, or services. However, depending on how you wish to use the service, you will need to launch the required drivers for sensors and actuators, and services as well. See the section Initialization, for more information.

Output

sensors: none
actuators: speakers, browser, see Pepper/Nao robots - restructured

Page layout

This interface allows executing instruction(s) on a webpage. The wizard-of-oz interface is horizontally divided into two parts as shown below:

Figure 1

Control: This section contains a selection of some basic interaction functionalities. A brief description of the following buttons is as follows:
- Start feed: Provides a live feed from the web sockets of audio and video, if the respective services are running
- Stop feed: Always use this before exiting the page to close the live feed correctly.
- Toggle idle: Locks or unlocks the autonomous mode of a Pepper or Nao robot.
- Eye color: Opens a pop-up in which one of the built-in eye LED colors can be selected for a Pepper or Nao robot
- Gesture: Opens a similar pop-up as Eye color with a selection of all available built-in animations on Pepper or Nao robots.
- Say: TTS on the robot (or other speaker devices)
- Say (anim): TTS on the robot with an animation
- Show (browser): show the text on a browser device (like Pepper’s tablet)
  - requires webserver

Favorites:
- customise the interface through a JSON configuration file
- use any set of individual actions
- for an overview of the available actions and their corresponding data, see Available Actions
For instance, figure 2 shows a set of some basic interaction functionalities. To view the configuration of the given example, please refer to Initialization → JSON file configuration.

Initialization

Using the service

Start the wizard service via docker-compose up wizard.
- You may run any other service you may require, for instance, docker-compose up wizard strean_audio stream video.
- You may run any drivers you might require (Only required when using a laptop’s sensors and actuators)
Navigate to your local IP address in a browser (or 192.168.99.100 if you’re using Docker Toolbox).
When loading the page initially, it will show all Devices that are currently connecting to your framework. The first checkbox can be used to check or uncheck all devices.
- At most one device per type should be selected (e.g. one robot and/or one camera), after which the Connect button should be pressed.

The service should now be running.

JSON file configuration

The JSON file should be an array of string objects:

header
- text - header title
- (optional) page
  - by default, each element is on page default; it can be assigned another page by changing the name of the page
button
- generating a button with a certain name
- execute an action with some preset data
  - (optional) have on of the style from: [primary, secondary, success, danger, warning, info, light, dark]) to change its appearance
  - (optional) execute multiple actions with one button by adding additional parameters action_1 and data_1, action_2 and data_2 etc.
  - (optional) {0}; {1}; {2}; etc. for taking input and replacing the button output accordingly
    - the same parameter (e.g.: {0}) can be used more than once in the data string
- (optional) page
- (optional) next to switch to another page upon press

Make sure that the JSON file is valid (using e.g. http://json.parser.online.fr).

Example

An example of a JSON configuration file can be found below:

[
  {
    "type": "header",
    "text": "Hello Goodbye"
  },
  {
    "type": "button",
    "style": "success",
    "name": "SayHello",
    "action": "action_say_animated",
    "data": "Hello, {0}!"
  },
  {
    "type": "button",
    "style": "danger",
    "name": "SayBye",
    "action": "action_say_animated",
    "data": "Goodbye!"
  },
  {
    "type": "button",
    "name": "Sing",
    "action": "action_say",
    "data": "You say yes, I say no, you say stop, and I say go, go, go. You say goodbye, and I say hello.",
    "next": "page_2"
  },
  {
    "type": "header",
    "text": "Other Stuff",
    "page": "page_2"
  },
  {
    "type": "button",
    "name": "Wave",
    "action": "action_gesture",
    "data": "animations/Stand/Gestures/Hey_1",
    "page": "page_2"
  },
  {
    "type": "button",
    "name": "Rainbow",
    "action": "action_eyecolour",
    "data": "rainbow",
    "next": "default",
    "page": "page_2"
  },
  {
    "type": "button",
    "style": "warning",
    "name": "Tablet",
    "action": "render_html",
    "data": "<img src=\"https://www.softbankrobotics.com/emea/sites/default/files/images-content/Pepper_8.png\">",
    "page": "page_2"
  }
]

The configuration file would be rendered to the pictures below:

the button “SayHello” would prompt for the name of the user input, so that “Hello, [name]” is used in TTS
upon press, the button “Sing” would trigger its singing action and then switch to page_2

default page

Events

Events are generated depending on the services used. For more information, check the respective service page.

Known Issues

There are no known issues.