Dissecting Face Detection Application

The purpose of this page is to layout exactly what elements of SIC are being run and where, why, and how, to gain more insight into the design of the framework. It is not to analyze how face detection works.

I will break it down into different levels of increasing detail as to not be too overwhelming at once. In this simple example, a desktop computer uses its camera to detect and display bounding boxes around faces.

Level 1 (Components, Connectors, and Managers)

1. FaceDetection service is ran in a separate shell, creating an associated ComponentManager

run-face-detection

within a shell

This runs the main() function of face_detection.py, which starts a ComponentManager assigned to the FaceDetectionComponent. In this example, this script is ran on the same computer as the rest of the application, although it may be ran on another computer.

2. Desktop device is instantiated

# Connect to the services
desktop = Desktop(camera_conf=conf)

within desktop_camera_facedetection.py

Rather than running a ComponentManager in a separate terminal, the Desktop class (inherits from Device) creates a new thread where its ComponentManager runs.

3. FaceDetection is connected to

face_rec = FaceDetection(ip="")

This instantiates a Connector which tries to connect to the actual FaceDetection component running on the specified IP.

4. Desktop camera feed is connected to FaceDetection

face_rec.connect(desktop.camera)

This connects the output channel of the camera to the input channel of the FaceDetection component. desktop.camera is also a Connector.

5. Callback functions are registered for Desktop camera and FaceDetection component

imgs_buffer = queue.Queue(maxsize=1)
faces_buffer = queue.Queue(maxsize=1)


def on_image(image_message: CompressedImageMessage):
    imgs_buffer.put(image_message.image)


def on_faces(message: BoundingBoxesMessage):
    faces_buffer.put(message.bboxes)
    
    
desktop.camera.register_callback(on_image)
face_rec.register_callback(on_faces)

For every message the DesktopCamera or FaceDetection components publish on their output channels, they also will put in their respective buffers.

6. Buffers are continuously read from, bounding boxes are drawn on and displayed

while True:
    img = imgs_buffer.get()
    faces = faces_buffer.get()

    for face in faces:
        utils_cv2.draw_bbox_on_image(face, img)

    cv2.imshow("", img)
    cv2.waitKey(1)

Assumes that the first image in the faces buffer (contains bounding boxes) corresponds with the first image in the images buffer.

Diagram of this level:

Takeaways:

• Components like FaceDetection do not run just by themselves. They consist of a Connectorwhich is basically a remote control for the actual component, a ComponentManager which runs them, and then the component itself.

Level 2 (Redis Channels)

1. FaceDetection service is ran in a separate shell, creating an associated ComponentManager

ComponentManager registers a request handler on a channel with the same name as its IP. This means whenever a message is published on this channel, it will process the message according to its function _handle_request (see ComponentManager script). As of writing this, it only expects two types of requests: it either starts a specific component given a SICStartComponentRequest, or sets a stop event to stop all of its associated components given a SICStopRequest.

2. Desktop device is instantiated

Same as above, except rather than being its own process this ComponentManager is in a thread belonging to the main application process. Moreover, it also manages a different set of components (microphone, camera, speakers, TTS) than the FaceDetection ComponentManager.

3. FaceDetection is connected to

The Connector stores the names of the following Redis channels (does not necessarily use them yet)

• input_channel: “FaceDetectionComponent:input:{IP}”

• _request_reply_channel: “FaceDetectionComponent:reqreply:{IP}”

• output_channel: “FaceDetectionComponent:{IP}”

Initially tries to ping the actual component. Since the actual component has not been started yet, it publishes a message on Redis which is then picked up by its ComponentManager. The associated ComponentManager then starts the actual component.

In this example, there are two ComponentManagers running on the same host. However, they have different sets of components. In the _handle_request function, each ComponentManager will check to see if the component that’s being requested belongs to it before it starts it. This way, multiple instances of the same component are not started if multiple ComponentManagers are running on the same host.

The connector then subscribes to the input channel so that messages can be manually sent if needed. **not exactly sure why we need this or if it is used.

4. Desktop camera feed is connected to FaceDetection

In the following line:

face_rec.connect(desktop.camera)

A component (desktop.camera) is passed into the connect() method of a Connector.

Level 3 (Multiple Users)