#RaspberryPi – Performance differences in #FaceRecognition using #OpenVino (code with @code!)

Buy Me A Coffee

Hi !

I’ve been looking to use the amazing Intel Neural Stick 2 for a while, and one of the 1st ideas that I have was to check how fast my Raspberry Pi 4 can run using this device.

The Intel team released a nice step by step process installation for Raspberry Pi. And it works great, there are a couple of minor glitches that you need to figure out, like the latest package version, everything else works great.

Note: I downloaded my openvino toolkit from here (https://download.01.org/opencv/2019/openvinotoolkit/R3/), and the downloaded file is (l_openvino_toolkit_runtime_raspbian_p_2019.3.334.tgz).

Once installed, the 1st python sample is a face recognition one. This sample analyzes a image file using OpenCV to detect faces, and creates a new output file with the detected images. As I said, is very straight forward.

So, I decided to create a new python sample to run live face detection using the camera feed and also display the FPS. This is the output code:

# perform face detection
# display detected face frame
# display FPS info in webcam video feed
# This is the official sample demo file desribed in the installer documentation
# Date: 2020 01 26
# Install OpenVINO™ toolkit for Raspbian* OS
# http://docs.openvinotoolkit.org/2019_R1/_docs_install_guides_installing_openvino_raspbian.html
import cv2
import time
import imutils
# Load the model.
net = cv2.dnn.readNet('face-detection-adas-0001.xml',
'face-detection-adas-0001.bin')
# Specify target device.
# ERROR net.setPreferableBackend(cv2.dnn.DNN_BACKEND_OPENCV)
# OK net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
# ERROR net.setPreferableBackend(cv2.dnn.DNN_BACKEND_HALIDE)
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)
# open video frame
video_capture = cv2.VideoCapture(0)
while True:
start_time = time.time()
ret, frame = video_capture.read()
# frame resize to improve performance
frame = imutils.resize(frame, width=648, height=480)
rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
# Prepare input blob and perform an inference.
blob = cv2.dnn.blobFromImage(rgb_frame, size=(640, 480), ddepth=cv2.CV_8U)
net.setInput(blob)
out = net.forward()
# Draw detected faces on the frame.
for detection in out.reshape(1, 7):
confidence = float(detection[2])
xmin = int(detection[3] * frame.shape[1])
ymin = int(detection[4] * frame.shape[0])
xmax = int(detection[5] * frame.shape[1])
ymax = int(detection[6] * frame.shape[0])
if confidence > 0.5:
cv2.rectangle(frame, (xmin, ymin), (xmax, ymax), color=(0, 255, 0))
#display FPS
fpsInfo = "FPS: " + str(1.0 / (time.time() start_time)) # FPS = 1 / time to process loop
print(fpsInfo)
font = cv2.FONT_HERSHEY_DUPLEX
cv2.putText(frame, fpsInfo, (10, 20), font, 0.4, (255, 255, 255), 1)
cv2.imshow('Video', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
video_capture.release()
cv2.destroyAllWindows()

The code is very straight forward and the main matters are

  • It uses 2 models from the Intel Zoo to perform the face detection: face-detection-adas-0001.xml and face-detection-adas-0001.bin
  • Lines 22 and 23 are key to define that OpenCV will load and use the models in the Intel device
  • I use imutils to resize the image to 640×480. Feel free to use any other library for this, even OpenCV
  • Also, it works also with smaller resolutions, however 640×480 is good for this demo

And the final app running analyzing almost 8 frames per second (8 FPS).

Which is almost 10 times faster that the 0.7 FPS without Intel NCS2

And, I already wrote about running Visual Studio Code in the Raspberry Pi (see references) is an amazing experience. I did all my Python in VSCode coding remote accesing my device via VNC. Python runs like a charm!

You can download the code from https://github.com/elbruno/rpiopenvino/tree/master/facedetection

References

My posts on Raspberry Pi

Dev posts for Raspberry Pi
Tools and Apps for Raspberry Pi
Setup the device
Hardware

#GITHUB – #Intel released the #SourceCode of Stephen Hawking’s speech recognition app

Hola!

When you are on holidays you must face the hard choice: run 10K or get into some amazing piece of source code. This one is very interesting, is Stephen Hawking’s speech recognition app developed by INTEL. The app name is Assistive Context-Aware Toolkit (ACAT), and as you probably already know it was created for Stephen Hawking who is affected by LS (amyotrophic lateral sclerosis), The main goal of the app is perform a face analysis and generate text based on facial gestures.

The main source code of the app is on GitHub (link). And you can download the code very in a couple of clicks. However I strongly recommend spend some time with the User Guide (link), so you can understand the prerequisites needed and the configuration steps.

The 2 main Options are download the app or download the source code. Previous image shows my choice.

Now I need to find some time to dig into the app, and probably write some posts.

So far, I read and understand that the app has 3 main components:

– The app perform face analysis using a webcam, that’s the entry point to start the “word translation from face gestures”

– There is a predictive engine to help in the Word autocomplete

– There is also a virtual keyboard (?)

I can quickly find 2 of them in the Source Code. And, of course, this is not an “everyone’s app”. The main purpose of put the source code available, is to get support from the community to improve the app. Sai Prasad said:

“Our hope is that, by open sourcing this configurable platform, developers will continue to expand on this system by adding new user interfaces, new sensing modalities, word prediction and many other features”

Saludos @ Malaga

/El Bruno

References

– Stephen Hawking’s speech software is now available for free http://www.theverge.com/2015/8/18/9173041/stephen-hawking-intel-github-speech-software

– Intel just open sourced Stephen Hawking’s speech system and it’s a .NET 4.5 Winforms app that you can try for yourself http://blogs.msdn.com/b/cdndevs/archive/2015/08/14/intel-just-open-sourced-stephen-hawking-s-speech-system-and-it-s-a-net-4-5-winforms-app.aspx

– ACAT Release page https://github.com/01org/acat/releases

– ACAT User Guide https://01.org/sites/default/files/documentation/acat_user_guide.pdf

#GITHUB – #Intel ha liberado el #SourceCode de la app de speech recognition que usa Stephen Hawking

Hola!

En vacaciones llegan estas noticias que te ponen frente a ti la difícil decisión de correr 10K o ponerse a estudiar un poco de source code del bueno. En este caso es más que interesantes, ya que es la app que usa Stephen Hawking’s para hacer algo parecido a un "speech recognition". El nombre de la app es Assistive Context-Aware Toolkit (ACAT) y está especialmente diseñada para personas con ALS (amyotrophic lateral sclerosis), donde con pocos movimientos es posible realizar un análisis de contexto, interpretar gestos y traducir esta información a texto.

Intel ha liberado el código de la app en GitHub (link). Descargarlo es bastante simple, aunque lo primero es darle un vistazo al User Guide (link), ya que hay que instalar un par de prerrequisitos y ejecutar unos pasos de configuración.

Las opciones son ejecutar la app o descargar y utilizar el código fuente, la imagen anterior muestra el camino que he seguido. Ahora me toca entender un poco lo que hace la app, y ver si saco tiempo para escribir al respecto.

Por lo que he leído, la app tiene 3 componentes principales:

– utiliza una webcam para detectar movimientos faciales con los que puede comenzar a interpretar las “palabras”

– tiene un motor predictivo que permite realizar un autocompletado de estas palabras

– existe un teclado virtual

2 de ellos pueden identificarse rápidamente en el Source Code.

Obviamente, esta app no es para un uso doméstico ni mucho menos, pero parafraseando a Sai Prasad la idea es expandir el core del sistema con nuevas UIs, nuevos modelos predictivos, etc.

“Our hope is that, by open sourcing this configurable platform, developers will continue to expand on this system by adding new user interfaces, new sensing modalities, word prediction and many other features"

Saludos @ Málaga

/El Bruno

References

– Stephen Hawking’s speech software is now available for free http://www.theverge.com/2015/8/18/9173041/stephen-hawking-intel-github-speech-software

– Intel just open sourced Stephen Hawking’s speech system and it’s a .NET 4.5 WinForms app that you can try for yourself http://blogs.msdn.com/b/cdndevs/archive/2015/08/14/intel-just-open-sourced-stephen-hawking-s-speech-system-and-it-s-a-net-4-5-winforms-app.aspx

– ACAT Release page https://github.com/01org/acat/releases

– ACAT User Guide https://01.org/sites/default/files/documentation/acat_user_guide.pdf