Dissecting Face Detection Application

The purpose of this page is to layout exactly what elements of SIC are being run where and how, to gain more insight into the design of the framework. It is not to analyze how face detection works.

I will break it down into different levels of increasing detail as to not be too overwhelming at once. In this simple example, a desktop computer uses its camera to detect and display bounding boxes around faces.

Level 1

1. FaceDetection service is ran in a separate shell, starting an associated ComponentManager

This runs the main() function of face_detection.py, which starts a ComponentManager assigned to the FaceDetectionComponent. In this example, this script is ran on the same computer as the rest of the application, although it may be ran on another computer.

2. Desktop device is instantiated

The Desktop class (inherits from Device) creates a new thread where its ComponentManager runs.

3. FaceDetection is connected to

This instantiates a Connector which tries to connect to the actual FaceDetection component running on the specified IP.

4. Desktop camera feed is connected to FaceDetection

This connects the output channel of the camera to the input channel of the FaceDetection component. desktop.camera is also a Connector.

5. Callback functions are registered for Desktop camera and FaceDetection component

For every message the DesktopCamera or FaceDetection components publish on their output channels, they also will put in their respective buffers.

6. Buffers are continuously read from, bounding boxes are drawn on and displayed

Assumes that the first image in the faces buffer (contains bounding boxes) corresponds with the first image in the images buffer.

Diagram of this level:

Open

Blue Hexagon represents the actual SIC application. Dotted arrows represent two way connections, whereas solid arrows are one-way.

Takeaways:

• Components like FaceDetection do not run just by themselves. They consist of a Connectorwhich is basically a remote control for the actual component, a ComponentManager which runs them, and then the component itself.

• Connectors send the actual requests to ComponentManagers to start/stop components and tell components to start listening to certain channels.

Level 2

1. FaceDetection service is ran in a separate shell, starting an associated ComponentManager

The ComponentManager registers a request handler on a Redis channel with the same name as its IP. This means whenever a message is published on its channel, it will process the message according to its function _handle_request (see ComponentManager script). As of writing this, it only expects two types of requests: it either starts a specific component given a SICStartComponentRequest, or given a SICStopRequest, sets a stop event to stop all of its associated components.

2. Desktop device is instantiated

Same as above, except rather than being its own process this ComponentManager is run in a thread belonging to the main application process. Moreover, it also manages a different set of components (microphone, camera, speakers, TTS) than the FaceDetection ComponentManager.

3. FaceDetection is connected to with FaceDetection Connector

The Connector stores the names of the following Redis channels:

• input_channel: “FaceDetectionComponent:input:{IP}”

• _request_reply_channel: “FaceDetectionComponent:reqreply:{IP}”

• output_channel: “FaceDetectionComponent:{IP}”

Initially tries to ping the actual component. Since the actual component has not been started yet, it publishes a message on Redis which is then picked up by its ComponentManager. The associated ComponentManager then starts the actual component.

In this example, there are two ComponentManagers running on the same host. However, they have different sets of components. In the _handle_request function, each ComponentManager will check to see if the component that’s being requested belongs to it before it starts it. This way, multiple instances of the same component are not started if multiple ComponentManagers are running on the same host.

4. Desktop camera feed is connected to FaceDetection

In the following line:

Desktop.camera is passed into the connect() method of a Connector. In the connect() method, the FaceDetection connector stores the output channel of the desktop.camera component in a ConnectRequest and publishes it on the request/reply channel of the actual FaceDetection component.

The _handle_request function of the actual FaceDetection component then picks up the request. It calls its _connect method which adds the output channel of desktop.camera to its list of input channels. It also registers its _handle_message function with Redis on this channel.

So now the FaceDetection component is subscribed to the output channel of the desktop.camera, and every time it receives a message on this channel, it runs the message through its _handle_message function.

5. Callback functions are registered for Desktop camera and FaceDetection component

The register_callback() Connector method is called for both desktop.camera and FaceDetection. The register_callback() function takes the passed in function, and uses Redis to subscribe the function to the output channel of its respective Component. So in this case, every time the FaceDetection finishes handling a message (where it creates a bounding box message), it publishes it on its output channel, and the callback function places it in the corresponding buffer within the application.

6. Buffers are continuously read from, bounding boxes are drawn on and displayed

This is the same.

Open

Red ‘RC’ circles are Redis connections, i.e. SICRedis instances that each element has. In addition to providing an API, each SICRedis stores different threads. Each thread consists of a handler function that acts on specified channel(s).

Takeaways:

• Components do not actually send message to each other directly, but rather publish messages/requests on Redis channels which are seen by other components that are subscribed to those channels.

• Redis connections create separate threads where message and request handlers run.

Level 3 (Multiple Users)