The social interaction cloud has many components available for you to speed up creating social interactions with the robot’s.
Depth estimation using the stereo grayscale cameras
Service | Command | Source | Demo files | Install | Note |
|---|---|---|---|---|---|
Dialogflow for creating conversational agents using google’s framework. This provides a flow chart like dialog management and speech recognition. |
| demo_desktop_microphone_dialogflow.py or demo_nao_dialogflow.py |
| |||
Face detection using OpenCV’s cascading classifier, which is very fast and can run on a laptop CPU |
| None, no extra dependencies are needed | ||||
DNN Face detection using a YOLOv7 neural network for accurate detection, and detection of small faces. |
|
| The model file used in this example can be found here: https:// |
DNN Face recognition using a resnet50 network to extract face embeddings and assign an id based on automatic clustering. |
|
Object tracking general object tracking using motpy for temporal knowledge of objects [WIP]
| The cascade classifier file used in this example can be found here: haarcascade_frontalface_default.xml. The model file can be found here: resnet50_ft_weight.pt | ||||
OpenAI ChatGPT a text based large language model that provides a very capable dialog agent. Requires a credit card. |
|
| An openai api key can be created here: https:// |
OpenAI Whisper a powerful speech to text model, capable of running both local and in the cloud. Cloud usage requires a credit card. Start and end recognition is performed using python’s https://pypi.org/project/SpeechRecognition/ |
|
| An openai api key can be created here: https://platform.openai.com/api-keys | ||
Google Text to speech using google cloud API. Requires a credit card. |
https://bitbucket.org/socialroboticshub/framework/src/master/sic_framework/services/text2speech/
Pepper’s Tablet for displaying URLs or HTML on the tablet.
(WIP) | |||||
Templates for creating your own components |