Tablets/Phones/Browsers

Owned by Vincent Koeman (Unlicensed)
Last updated: Mar 29, 2022
2 min read

You can connect any browser to the SIC framework (with a running https://socialrobotics.atlassian.net/wiki/spaces/CBSR/pages/1470070884 service) by navigating to https://<serverIP>:11880. You might need to ignore/circument warnings about a self-signed SSL certificate being used. On Java-capable devices, the computer-browser program can do this automatically. On Pepper robots, the tablet is automatically connected as well, although a pop-up will need to be dismissed on the tablet upon first connection. In any case, the browser will be automatically registered as a device, showing the identifier upon load. Note that this identifier will be different each time the page reloads (there is no mechanism for permanent identifiers in JavaScript).

By default, the Bootstrap framework will be loaded (v4.6), and can thus be used to render a styled HTML body from an application (a websocket-based communicaton protocol is used for this). Any <button> element will automatically send its contents to the application. In addition, giving one of the following classes to an element (e.g. a <div>) has built-in effects:

  • chatbox: shows a text-type input, from which the input is sent to the Dialogflow service upon submission.

  • english_flag: shows an English flag, which when clicked upon will send a command to set the language to 'en-US'.

  • listening_icon: shows a listening icon (in the form of a microphone), which shows a user when the microphone is open.

  • speech_text: shows a live transcript of the text currently recognised by the Dialogflow service.

  • vu_logo: shows a VU logo.

Tip: custom images (i.e. that don’t have a public URL) can be embedded using Base64 encoding.

Adding the audioEnabled class somewhere in the content will allow the browser device to also function as a full-fledged microphone and speaker device; any audio-related action in the framework will be handled! Note that permission might have to be given to enable for instance the microphone, and that usually some user interaction (like pressing a button) is required before being allowed to play audio. Not all devices and/or browsers support this either (e.g. on iOS Safari it does not work at this time), and there is a limit of 200 characters on the (Google) text-to-speech that is used in the browser.