What are SIC Components?
SIC provides components to combine different robot’s, online services and data processing tools. SIC Components are the core building blocks of the framework. A component can receive some form of input, perform processing and possibly provide an output.
You can think of components as a sort of LEGO blocks. To program the robot, we combine a couple of these blocks to provide a specific functionality. To make the robot greet a person walking up to it for example, we’ll need to combine a camera component, face detection component and a text-to-speech component.
We can show this in a more schematic approach. For example, here are the building blocks to get started in detecting a person walking up to the robot. The device or robot that is running the component code is shown in italics below the component name.
Notice that your own code, the SIC program, is shown as a component. This is not actually a component, but can receive inputs and provide outputs, and can therefore also be seen as a component.
Here is the code to first create the two components:
nao = Nao(ip="192.168.0.148")
face_rec = FaceDetection()Showing the robot’s camera stream
On their own, components don’t do very much. Lets make it interesting by connecting them! We can connect to the Nao robot’s camera like this:
The code to connect the CameraSensor component to our SIC program is the following (especially line 8):
imgs_buffer = queue.Queue()
def on_image(image_message: CompressedImageMessage):
imgs_buffer.put(image_message.image)
nao = Nao(ip="192.168.0.148")
face_rec = FaceDetection()
nao.register_callback(on_image)
while True:
img = imgs_buffer.get()
cv2.imshow("robot's view", img)
cv2.waitKey(1)Notice here the line nao.register_callback(on_image). This call connects us to the stream of images from the robot’s camera. Every time we receive a new image, the on_image function will be called. We put this in a queue, because the on_image function will be called in another thread. To get the data back in the main program thread, we put it in the queue and in the main program loop we use .get(), which blocks until new data is available. This way, we know for sure that the img variable will contain an image!
Detecting a face in an image
Alright, so we have images now. How do we detect faces in the image? Lets take a small step back, and focus on detecting faces on a single image first. Detecting faces on a stream of images is better done in a different way, but we’ll get to that.
For detecting faces in a single image, we’ll use something called a Request. You can send a request to a component, and it replies with a Message. This is useful, as the bounding boxes we get back are specifically for the image we sent.
In code, this looks like this:
buffer = queue.Queue()
def on_image(bbox_message: BoundingBoxesMessage):
buffer.put(bbox_message)
face_rec = FaceDetection()
img = cv2.imread('/path/to/image/some_image.png')
image_request = CompressedImageRequest(image=img)
boundingboxes = face_rec.request(image_request)
for bbox in boundingboxes.bboxes:
print("Face at", bbox.x, bbox.y)Notice that, on line 6, we create the FaceDetection component again. Then, on line 11, we send a request from our SIC program to the component and receive a reply (boundingboxes).
Details
Messages and requests
A component can implement two callbacks, on_message() and on_request().
A request must be met with a reply
Technical details
Redis server
The IP can be set using export DB_IP=192.168.0.181.
Defaults to localhost.
Redis channel specification
output channel format:
{name}:{ip} e.g. DialogflowService:192.168.0.181
request-reply channel format:
{name}:reqreply:{ip} e.g. DialogflowService:reqreply:192.168.0.181
Knowing the IP of the device and the service we expect is running, the request reply channel and output channel are known. Using the request reply channel we can request the component to start listening to input channels (using a ConnectRequest).
- Deterministic channel names simplify the structure of the framework significantly. Communication with the component is optional, as one can simply assume it is running on the remote device and listen to the output channel or send requests. There are some consequences
1. Only one instance of a service can run on a device at once
Control requests and messages
To control a running component, system level requests can be sent. These requests will not be passed on to the message handlers (on_request and on_message). These requests should inherit from SICControlRequest.
SICPingRequest- A ping message to check if a component is active. Should be answered with a reply by sending back aSICPongMessage.ConnectRequest- A request for a component to start listening to some some channel. The messages on this channel will be passed onto theon_messagecallback.SICSuccessMessage- A message indicating succes, for example responding to aConnectRequest.SICStopRequest- A request for the component to shut down.SICIgnoreRequestMessage- A message responding that the request is not accepted. TODO Deprecate?