Versions Compared

Old Version 7

changes.mady.by.user Thomas Wiggers

Saved on

compared with

New Version Current

changes.mady.by.user Karen Chiang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The social interaction cloud has many components available for you to speed up creating social interactions with the robot’s.

[WIP] Depth estimation using the stereo grayscale cameras

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/depth_estimation/

Service

Command

Source

Demo files

Install

Note

Dialogflow for creating conversational agents using google’s framework. This provides a flow chart like dialog management and speech recognition.

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/dialogflow/

run-dialogflow

dialogflow

demo_desktop_microphone_dialogflow.py or demo_nao_dialogflow.py

pip install social-interaction-cloud[dialogflow]

Face detection using OpenCV’s cascading classifier, which is very fast and can run on a laptop CPU

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/face_detection/

run-face-detection

face_detection

demo_desktop_camera_facedetection.py

None, no extra dependencies are needed

DNN Face detection using a YOLOv7 neural network for accurate detection, and detection of small faces.

run-face-detection-dnn --model yolo7-face.pt

face_detection_dnn

demo_desktop_camera_facedetection_dnn.py

pip install social-interaction-cloud[face-detection-dnn]

The model file used in this example can be found here: https://

bitbucket

drive.google.

org

com/

socialroboticshub

file/

framework

d/

src/master/sic_framework/services/face_detection_dnn/

1oIaGXFd4goyBvB1mYDK24GLof53H9ZYo/view

DNN Face recognition using a resnet50 network to extract face embeddings and assign an id based on automatic clustering.

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/

run-face-recognition --model xxx.pt --cascadefile xxx.xml

face_recognition_dnn

/

[WIP] Object tracking general object tracking using motpy for temporal knowledge of objects

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/object_tracking/

demo_desktop_camera_facerecognition.py

pip install social-interaction-cloud[face-recognition]

The cascade classifier file used in this example can be found here: haarcascade_frontalface_default.xml. The model file can be found here: resnet50_ft_weight.pt

OpenAI ChatGPT a text based large language model that provides a very capable dialog agent. Requires a credit card.

run-gpt

gpt

demo_openai_gpt.py

pip install social-interaction-cloud[openai-gpt]

An openai api key can be created here: https://

bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/openai_gpt/

platform.openai.com/api-keys

OpenAI Whisper a powerful speech to text model, capable of running both local and in the cloud. Cloud usage requires a credit card. Start and end recognition is performed using python’s https://pypi.org/project/SpeechRecognition/

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/openai_

run-whisper

whisper_speech_to_text

demo_desktop_microphone_whisper.py

pip install social-interaction-cloud[whisper-speech-to-text]

An openai api key can be created here: https://platform.openai.com/api-keys

Google Text to speech using google cloud API. Requires a credit card.

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/text2speech/

Pepper’s Tablet for displaying URLs or HTML on the tablet.

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/webserver/

(WIP)

text2speech

Templates for creating your own components

https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/

templates

/