Page Comparison

This guide introduces the main components and concepts of this API and provides some code samples that can be used as a starting point.

Requirements

The Python-SIC connector requires Python 3 (with tkinter enabled).
1. Make sure Python can compile native extensions (e.g. for Windows see https://www.scivision.dev/python-windows-visual-c-14-required).
2. You can use a Python editor of your choice (Pycharm for example).

The Python skeleton project itself can be found in the ‘python’ folder at https://bitbucket.org/socialroboticshub/connectors. It is recommended to simply download this repository or even fork it directly (which you can do using the + button at the top-left of the page). Once you have your download or cloned fork, make sure the required dependencies are installed

...

Installation

Code Block
pip install social_interaction_cloud

Resources

API Documentation Python Connector API

Documented examples https://bitbucket.org/socialroboticshub/examples/src/main/

Background information Python Connector Background Information

action commands
- use the methods available in the SIC connector
- allow you to create your application using the SIC framework
data generated
- contains events (e.g.: when a button is pressed LeftBumperPressed or when an action is finished WakeUpDone)
- the results of certain actions (e.g.: a recognised recognized intent after a speech recognition attempt)

...

In the below MyConnector example, you see that it uses the AbstractSICConnectorclass as a parent, inheriting all its methods. Two things have been added:

the on_robot_event method is overridden to print all the events generated by the robot
the run method sends actions to the SIC

Code Block

breakoutMode	wide

from social_interaction_cloud.abstract_connector import AbstractSICConnector
from time import sleep


class MyConnector(AbstractSICConnector):
    def __init__(self, server_ip):
        super(MyConnector, self).__init__(server_ip)

    def run(self):
        self.start()
        self.set_language('en-US')
        sleep(1)  # wait for the language to change
        self.say('Hello, world!')
        sleep(3)  # wait for the robot to be done speaking (to see the relevant prints)
        self.stop()

    def on_robot_event(self, event):
        print(event)


# Run the application
my_connector = MyConnector(server_ip='127.0.0.1')
my_connector.run()

The working of the example is as follows:

run
1. self.start() activates the connection.
  1. Under the hood, a thread is started allowing the connector to receive actions
2. self.set_language('en-US') and self.say('Hello, world!') are the two actions sent to the robot to make it say ‘Hello, world!’ in English.
3. self.stop() gracefully closes the connection.
4. The sleep statements avoid the program to stop before all the events are generated.
  1. See what happens when you remove the sleep statements. Most of the time you do not know how long you have to wait for an action to finish. Therefore, sleep statements are not the way to go. Ideally, you want the device to wait until it has received the necessary data and select it’s next action based on the available data
on_robot_event
1. The on_robot_event method will print all incoming events, which are: LanguageChanged, TextStarted, and TextDone. If you, for example, touch a robot's head sensors (while the program is running), the events FrontTactilTouched, MiddleTactilTouched, and/or RearTactilTouchedwill also be printed.

Note: These methods, as are all actions, are asynchronous. This means that they do not wait for a result before continuing. It also allows, if supported by the connected device(s), to execute actions in parallel (e.g. simultaneously speaking and gesturing).

...

Code Block

breakoutMode	wide

import threading
from social_interaction_cloud.basic_connector import BasicSICConnector
from time import sleep


class Example:

    def __init__(self, server_ip):
        self.sic = BasicSICConnector(server_ip)

        self.awake_lock = threading.Event()

    def run(self):
        # active Social Interaction Cloud connection
        self.sic.start()

        # set language to English
        self.sic.set_language('en-US')

        # stand up and wait until this action is done (whenever the callback function self.awake is called)
        self.sic.wake_up(self.awake)
        self.awake_lock.wait()  # see https://docs.python.org/3/library/threading.html#event-objects

        self.sic.say_animated('You can tickle me by touching my head.')
        # Execute that_tickles call each time the middle tactile is touched
        self.sic.subscribe_touch_listener('MiddleTactilTouched', self.that_tickles)

        # You have 10 seconds to tickle the robot
        sleep(10)
        
        # Unsubscribe the listener if you don't need it anymore.
        self.sic.unsubscribe_touch_listener('MiddleTactilTouched')

        # Go to rest mode
        self.sic.rest()

        # close the Social Interaction Cloud connection
        self.sic.stop()

    def awake(self):
        """Callback function for wake_up action. Called only once.
        It lifts the lock, making the program continue from self.awake_lock.wait()"""

        self.awake_lock.set()

    def that_tickles(self):
        """Callback function for touch listener. Everytime the MiddleTactilTouched event is generated, this
         callback function is called, making the robot say 'That tickles!'"""
        
        self.sic.say_animated('That tickles!')


example = Example('127.0.0.1')
example.run()

In the example above:

A connected

...

NAO robot will stand up, saying “You can tickle me by touching my head”
1. To wait until the

...

1. NAO has finished standing up, the program is locked by the self.awake_lock.wait() statement.
  1. awake_lock is

...

1. 1. a threading.Event() object, that blocks the main thread until the threading.Event() is set by calling self.awake_lock.set(). This is done in the awake callback function. This callback function is added to the wake_up action.
2. Once the robot is finished standing up, awake is called, and the “lock is lifted”, allowing the program to continue.
For 10 seconds will say “that tickles” every time you touch the sensor on the middle of its head.
After 10 seconds, the

...

NAO will sit down again.

A different callback function is that_tickles. It is subscribed to the MiddleTactilTouched event. Whenever the program is running, that_tickles is called each time the middle head sensor is touched.

...

To ActionFactory helps build actions and especially can save you the trouble of managing all the different locks you might need. The following actions define the core functionality of ActionFactory:

The build_action method returns a regular action. But instead of providing a reference to the BasicSICConnector method, you can use its name.

Code Block
sic = BasicSICConnector(server_ip) action_factory = ActionFactory(sic) hello_action_factory = action_factory.build_action('say', 'Hello, Action Factory!') hello_action_factory.peform()

The

build_waiting_action method returns a waiting action. The ActionFactory creates a lock with a callback function that releases the lock for you. Optionally you can also add your own callback function, that will be embedded in the callback function created by the ActionFactory.

Code Block

breakoutMode	wide

wake_up_action = action_factory.build_waiting_action('wake_up', additional_callback=awake)
wake_up_action.perform().wait()

def awake():
  print('I am fully awake now')

The

build_touch_listener method lets you build a touch listener that is registered to the BasicSICConnector.

You can register to all the robot’s sensor events (e.g. MiddleTactilTouched).
You can use it to wait until a button is pressed or to do something when a button is pressed.
The listener can be notified only once or every time (until you unregister the listener) the button is pressed.
The build_vision_listener method works similarly to the build_touch_listener method. It lets you build a listener for four specific vision events: on_person_detected, on_face_recognized, on_emotion_detected, and on_corona_check_passed.
For these events to be generated you need to turn on the People Detection, Face Recognition, Emotion Detection, and Corona Checker services respectively. To build a vision listener for these three events, you the label ‘people’, ‘face’, ‘emotion’ or ‘corona’ respectively for the vision_type parameter.

Finally, the last component of the action package is the

ActionRunner.

It allows you to run regular and waiting actions right away with run_action and run_waiting_action respectively.
It takes the same input as the ActionFactory methods because that is being called under the hood first.
ActionRunner also allows you to preload a set of actions, run them in parallel, and wait for all of the waiting actions (not regular actions) to finish before continuing. Use load_waiting_action and load_waiting_action to preload actions and run_loaded_actions to run them.

...

To structure your code using state and state transitions you can use the state machine design pattern. See https://levelup.gitconnected.com/an-example-based-introduction-to-finite-state-machines-f908858e450f, https://medium.datadriveninvestor.com/state-machine-design-pattern-why-how-example-through-spring-state-machine-part-1-f13872d68c2d or this paper for a more extensive explanation of what they are.

...

Code Block

breakoutMode	wide

from transitions import Machine

class ExampleRobot(object):
   
   states = ['asleep', 'awake', 'introduced', 'got_acquainted', 'goodbye']

   def __init__(self):
      self.machine = Machine(model=self, states=ExampleRobot.states, initial='asleep')
      self.machine.add_transition(trigger='start', source='asleep', dest='awake',
                            before='wake_up', after='introduce')
      self.machine.add_transition(trigger='introduce', source='awake', dest='introduced',
                            before='introduction', after='get_acquainted')
      ...
      
   def wake_up(self):
      self.action_runner.load_waiting_action('set_language', 'en-US')
      self.action_runner.load_waiting_action('wake_up')
      self.action_runner.run_loaded_actions()

robot = ExampleRobot()
robot.start() # causes state transition from asleep to awake

define all the states of the state machine (line 5)
initialise the state machine with the model, state and initial state (line 8)
add transitions between states (line 9) - if we have an instantiation of the ExampleRobot class we can now call the start method (trigger) to cause a transition from the initial asleep state (source) the the awake state (destination):
- the trigger to a transition is the method that causes the transition
  - in this case, the robot will wake up upon the call of the start method (line 21)
- the source of a transition is the previous state in which the state machine was
  - in this case, “asleep”
- the destination of a transition is the state to which the transition directs the state machine is directed
  - in this case, “awake”
- before trigger of a transition is a statement to trigger a method before the transition happens
  - in this case, method wake_up (line 15)
- after trigger of a transition is a statement to trigger after a transition happens
  - in this case, “introduce” becomes a trigger to the next transition “introduced” (line 11)
  - note: Often there are no external triggers to trigger a state transition in the human-robot interaction flow. For example, when the robot is awake and ready it should automatically move to a next state. Adding an after parameter to the next transition as a trigger would address this issue

For a complete working example see https://bitbucket.org/socialroboticshub/connectorsexamples/src/mastermain/python/4_state_machine_example.py .

Versions Compared

Old Version 1

New Version Current

Key

Requirements

Installation

Resources

Table of Contents