The Social Interaction Cloud (SIC) is a light-weight, easy-to-use framework for developing socially interactive systems. It has been developed with the aim to facilitate social interaction with physical devices. The framework can be also used to easily create data pipelines and enables scaling to support more advanced architectures using more advanced cloud-based computing, for example.

API

Running a hello world on a NAOv6 robot is as simple as

from sic_framework.devices import Nao
from sic_framework.devices.common_naoqi.naoqi_text_to_speech import NaoqiTextToSpeechRequest

nao = Nao(ip='192.168.0.151')  # change into IP-address of your Nao robot.
nao.tts.request(NaoqiTextToSpeechRequest("Hello world!"))

Installation & Getting started

You can find the instructions on how to install the framework here: Getting started

To include a robot in the mix read Getting started with a robot

Components

Quickly link per-trained models and cloud solutions together using components, such as:

There are plenty of demo’s on how to use various components in https://github.com/Social-AI-VU/sic_applications, so check them out!

Overview of framework structure

To give an example of how the framework is structured, here are three usecases and the code a student would have to write

A student wants to display face recognition on their laptop.
1 input, 1 output to student device
A student wants to send and image to face recognition and save the result
1 input, 1 output that must be tied to the input
A student wants to send robot audio to dialogflow and wave when it detects “Hello”
2 asynchronous inputs (audio, command), 2 asynchronous outputs (transcript, intent)

Display face recognition

Do face recognition on a nao’s camera stream

image = None
def set_image_variable(img):
  image = img

bbox = None
def set_bbox_variable(box):
  bbox = box
  
nao = Nao(ip="192.168.0.181")  # change into IP-address of your Nao robot.
face_recognition = FaceRecognition(ip="127.0.0.1")

nao.top_camera.register_callback(set_image_variable)

face_recognition.connect(nao.top_camera)
face_recognition.register_callback(set_bbox_variable)

while True:
  image.draw(bbox)
  display(image)

Do a single face recognition

Recognize the faces in a picture from the student’s laptop

face_recognition = FaceRecognition()

image = load_from_disk("picture.jpg")

image_request = FaceRecognitonRequest(image=image)
bboxes = face_recognition.request(image_request)

draw_bbox_on_image(image.image, bboxes.bboxes[0])

display(image.image)

Dialogflow hello detection

A demo that has a robot wave whenever someone says “hello” and prints the detected transcript on the laptop.

pepper = Pepper(ip="192.168.0.123")  # change into IP-address of your Pepper robot.


keyfile_json = json.load(open("sail-380610-0dea39e1a452.json"))
conf = DialogflowConf(keyfile_json=keyfile_json,
                      sample_rate_hertz=16000, )
dialogflow = Dialogflow()

dialogflow.connect(pepper.microphone)
dialogflow.register_callback(print_transcript)

while True:
  intent = dialogflow.request(ListenForIntentRequest())
  if intent.name == "hello":
    wave_req = PepperMotionRequest("wave")
    pepper.motion.request(wave_req)

The Social Interaction Cloud (SIC) V2

API