#ComputerVision – Object Detection with #YoloV4 (work in progress …) and let’s think about ethics in Computer Vision

Buy Me A Coffee

Hi !

So after yesterday post where I used YoloV3 and MobileNetSSD, I also remember that we have YoloV4 released on April. I managed to make my code work with YoloV4 with some poor FPS results.

If you are interested on the code, let me know and I’ll be happy to share the code. It’s still a mess, working but a mess.

Abstract: There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy. Practical testing of combinations of such features on large datasets, and theoretical justification of the result, is required. Some features operate on certain models exclusively and for certain problems exclusively, or only for small-scale datasets; while some features, such as batch-normalization and residual-connections, are applicable to the majority of models, tasks, and datasets. We assume that such universal features include Weighted-Residual-Connections (WRC), Cross-Stage-Partial-connections (CSP), Cross mini-Batch Normalization (CmBN), Self-adversarial-training (SAT) and Mish-activation. We use new features: WRC, CSP, CmBN, SAT, Mish activation, Mosaic data augmentation, CmBN, DropBlock regularization, and CIoU loss, and combine some of them to achieve state-of-the-art results: 43.5% AP (65.7% AP50) for the MS COCO dataset at a realtime speed of ~65 FPS on Tesla V100.

However, what I also learned is part of the story behind YoloV4. This is very relevant to our days. The next 10 min video, really nails an explanation about how YoloV4 works.

YOLO History

YOLO was developed by Joseph Redmon. It was 1st presented in 2016, and it was key for object recognition research. This led to better and faster Computer Vision algorithms.

The latest version, YOLO v4 is currently developed by three developers:

  • Alexey Bochkovskiy
  • Chien-Yao Wang
  • Hong-Yuan Mark Liao

No Joseph Redmon in YOLOv4?

Joseph Redmon quit developing YOLO v4 because of the potential misuse of his tech. He recently announced that he would stop doing computer vision research because of the military and ethical issues….

So, why this is important? It’s all about how we use this technology. There are amazing advances in the Computer Vision area, but we also are lacking some regulation about how to use this.

IBM announced that they will no longer offer facial recognition software

2 days ago, IBM announced that they will no longer offer facial recognition software. The Verge wrote an amazing article about this (see references). This sentences really hit a point regarding Ethics and more:

IBM will no longer offer general purpose facial recognition or analysis software, IBM CEO Arvind Krishna said in a letter to Congress today. The company will also no longer develop or research the technology, IBM tells The Verge. Krishna addressed the letter to Sens. Cory Booker (D-NJ) and Kamala Harris (D-CA) and Reps. Karen Bass (D-CA), Hakeem Jeffries (D-NY), and Jerrold Nadler (D-NY).

“IBM firmly opposes and will not condone uses of any [facial recognition] technology, including facial recognition technology offered by other vendors, for mass surveillance, racial profiling, violations of basic human rights and freedoms, or any purpose which is not consistent with our values and Principles of Trust and Transparency,” Krishna said in the letter. “We believe now is the time to begin a national dialogue on whether and how facial recognition technology should be employed by domestic law enforcement agencies.” Facial recognition software has come under scrutiny for issues with racial bias and privacy concerns

Facial recognition software has improved greatly over the last decade thanks to advances in artificial intelligence. At the same time, the technology — because it is often provided by private companies with little regulation or federal oversight — has been shown to suffer from bias along lines of age, race, and ethnicity, which can make the tools unreliable for law enforcement and security and ripe for potential civil rights abuses.

The Verge, IBM will no longer offer, develop, or research facial recognition technology

There it is, think about this.

Happy coding!

Greetings

El Bruno

Resources

#ComputerVision – Object Detection with #YoloV3 and #MobileNetSSD

Buy Me A Coffee

Hi !

I have a ToDo in my list, to add some new drone demos. In order to do this, I was planning to perform some tests with pretrained models and use them. The 1st 2 in my list are Yolo and MobileNetSSD (see references).

YoloV3

Let’s start with one of the most popular object detection tools, YOLOV3. The official definition:

YOLO (You Only Look Once) is a real-time object detection algorithm that is a single deep convolutional neural network that splits the input image into a set of grid cells, so unlike image classification or face detection, each grid cell in YOLO algorithm will have an associated vector in the output that tells us:

If an object exists in that grid cell.

The class of that object (i.e label).

The predicted bounding box for that object (location).

YoloV3

I pickup some sample code from GitHub repositories and, as usual, from PyImageSearch (see references), and I created a real-time object detection scenario using my webcam as the input feed for YoloV3.

Object Detection live sample with Yolo V3

The final demo, works great; we can use the 80 classes that YoloV3 supports and it’s working at ~2FPS.

MobileNetSSD

Another very popular Object Detection Tool is MobileNetSSD. And, the important part here is SSD, Single Shot Detection. Let’s go to the definition:

Single Shot object detection or SSD takes one single shot to detect multiple objects within the image. As you can see in the above image we are detecting coffee, iPhone, notebook, laptop and glasses at the same time.

It composes of two parts

– Extract feature maps, and

– Apply convolution filter to detect objects

SSD is developed by Google researcher teams to main the balance between the two object detection methods which are YOLO and RCNN.

There are specifically two models of SSD are available

– SSD300: In this model the input size is fixed to 300×300. It is used in lower resolution images, faster processing speed and it is less accurate than SSD512

– SSD512: In this model the input size is fixed to 500×500. It is used in higher resolution images and it is more accurate than other models.

SSD is faster than R-CNN because in R-CNN we need two shots one for generating region proposals and one for detecting objects whereas in SSD It can be done in a single shot.

The MobileNet SSD method was first trained on the COCO dataset and was then fine-tuned on PASCAL VOC reaching 72.7% mAP (mean average precision).

For this demo, I’ll use the SSD300 model. Even, if the drone support better quality images and the SSD500 model works with bigger images, SSD300 is a good fit for this.

bject Detection with MobileNetSSD

This sample works at ~20FPS, and this triggered my curiosity to learn more about the 2nd one. I started to read a lot about this, and found some amazing articles and papers. At the end, if you are interested on my personal take, I really enjoyed this 30 min video about the different detectors side-by-side

Source Code

YoloV3 webcam live object detection

MobileNetSSD webcam live object detection

Happy coding!

Greetings

El Bruno

Resources

#WinML – Alternatives to #Yolo for object detection in #ONNX format

Hi!

A few days ago I commented with some colleagues the example of using TinyYolo In a UWP Application. Now it is a very task, because we can use a ONNX model in an Windows 10 application.

Note: The App can be an UWP app or a standard Win32 app, like, for example, the classic Windows forms.

Well, in the middle of the conversation, someone ask the following question, which is also a classic in ML events and talks:

Is there an AI model Marketplace for Windows?

The answer is yes. And although it’s not a Marketplace Models only, at Azure AI Gallery (Https://gallery.azure.ai/models) We can find many ONNX models, already ready to be used in Windows 10.

01 azure ai gallery home

On the main page we can see the different models organized by categories and by relevance. If we go into the detail of one of the models, we will see the details of the same, Like this As the option of downloading the file ONNX

02 azure ai gallery Alex Net details

Finally, according to the documentation of the model, we will be able to access your GitHub repository, Paper Where it explains the same, and other options.

03 AlexNet paper

So, you know, if you want to add some AI capabilities in your Apps, the Azure AI Gallery is a #MustReview Place to look!

Happy Coding!

Greetings @ Toronto

El Bruno

References

Windows 10 and YOLOV2 for Object Detection Series

#WinML – Alternativas a #TinyYolo para reconocimiento de objetos en formato #ONNX

Buenas!

Hace unos días comentaba con unos colegas el ejemplo de utilización de TinyYolo en una UWP. Ahora es muy simple poder utilizar un modelo de ML en formato ONNX y utilizarlo en una aplicación en Windows 10.

Nota: la app puede ser UWP o una app Win32 estándar, como, por ejemplo, los clásicos Windows Forms.

Pues bien, en el medio de la conversación, surgió la pregunta que mas respondo en eventos y charlas:

¿Hay un Marketplace de modelos de AI para Windows?

La respuesta es SI. Y aunque no es un marketplace solo de modelos, en Azure AI Gallery (https://gallery.azure.ai/models) podemos encontrar muchos modelos ONNX, ya preparados para ser utilizados en Windows 10.

01 azure ai gallery home

En la página principal podemos ver los diferentes modelos organizados por categorías y por relevancia. Si entramos al detalle de uno de los modelos, veremos los detalles del mismo, asi como la opción del descargar el archivo ONNX

02 azure ai gallery Alex Net details

Finalmente, de acuerdo con la documentación del modelo, podremos acceder a su repositorio de GitHub, al paper donde se explica el mismo, y otras opciones.

03 AlexNet paper

Así que, ya sabes, si quieres agregar algunas capacidades de AI en tus apps, la Azure AI Gallery es un #MustReview place donde buscar!

Happy Coding!

Saludos @ Toronto

El Bruno

References

Windows 10 and YOLOV2 for Object Detection Series

#WinML – Updated demo using Tiny YOLO V2 1.2, Windows 10 and YOLOV2 for Object Detection Series

Windows 10 and YOLOV2 for Object Detection Series


Hi!

There is a new Tiny YOLO V2 version in Azure AI Gallery

Tiny YOLOv2 1.2

I’ve updated my sample in GitHub to use this new version

https://github.com/elbruno/Blog/tree/master/20180806%20UwpMLNet%20TinyYoloV2%201.2

And it seems that Windows Insiders, are still having issues loading ONNXs models. My current build is 17730.1000

I1.png

Happy Coding!

Greetings @ Toronto

El Bruno

References

#WinML – Demo actualizada para utilizar Tiny YOLOv2 1.2, Windows 10 and YOLOV2 for Object Detection Series

Windows 10 and YOLOV2 for Object Detection Series


Buenas!

Ya tenemos disponible una nueva version de Tiny YOLO V2 en Azure AI Gallery

Tiny YOLOv2 1.2

Así que he actualizado el sample en GitHub para que utilice esta nueva version

https://github.com/elbruno/Blog/tree/master/20180806%20UwpMLNet%20TinyYoloV2%201.2

Y solo recordar que la carga de modelos ONNX parece que sigue sin funcionar en las versiones de Windows Insiders, en mi caso la build 17730.1000

I1.png

 

Happy Coding!

Saludos @ Toronto

El Bruno

References

#WinML – How to create a #Windows10 App and use #TinyYOLOV2 for object detection (the complete series)

Hi !

Windows 10 and TinyYOLOV2 for Object Detection Series

The complete example in GitHub

https://github.com/elbruno/Blog/tree/master/20180709%20UwpMLNet%20TinyYoloV3

Happy Coding!

Greetings @ Toronto

El Bruno

References

#WinML – Tutorial para convertir YoloV3 de CoreML a Onnx para utilizarlo en una #Windows10 App

Crear una Windows 10 UWP App y utilizar YoloV2 para reconocer objetos


Buenas !

En el post de hoy comentare como descargar la última versión de Tiny-YoloV3 y utilizarla en la UWP App que cree en post anteriores. Solo como reminder, la versión que utilice era Tiny-YoloV2 que es la que esta disponible en formato Onnx para descargar desde Azure AI Gallery.

Podemos descargar Tiny-YoloV3 desde su página oficial, sin embargo yo trabajare con una versión que ya está compilada en formato CoreML, que el formato de ML que se suele utilizar en apps iOS (ver referencias).

Pues bien, para convertir el modelo de CoreML a Onnx, utilizaremos Visual Studio Tools for AI, y el siguiente conjunto de software

Una vez instalado todo el software, podemos seguir el paso a paso de [AI Converting models to ONNX] para convertir nuestro modelo. Sin embargo, el camino no es tan simple como parece. Lo 1ro que nos podemos encontrar son errores como el siguiente

01 thanks Python

Problemas con Python, en mi caso tenia varias versiones de Python instaladas, pero el IDE no tenia ninguna marcada como [Default]. Desde el panel [Pythin Environments] se puede solucionar esto

01 1 Python environments

El siguiente problema que necesito un poco de configuración de mi parte, estaba relacionado con prerequisitos para la conversión. Todo comienza con [Missing package WinMLTools]

—————————

Error

—————————

Missing package WinMLTools, please check details in output window.

—————————

OK  

—————————

 

Traceback (most recent call last):

  File “C:\Users\<Bruno>\AppData\Local\Microsoft\VisualStudio\15.0_e5344afb\Extensions\kzqekf1z.44v\RuntimeSDK\model\model_converter_cli.py”, line 76, in check_winmltools_installed

    import winmltools

ModuleNotFoundError: No module named ‘winmltools’

Sin embargo, despues de investigar un rato, estas son los 2 packages que necesito instalar

Microsoft ML Tool (winmltools)

pip3 install winmltools==0.1.0.5072

CoreML Tools

pip3 install “git+https://github.com/apple/coremltools@v0.8”

Claro, antes hay que actualizar Python

03 upgrade pip.png

Y ya podemos lanzar nuevamente la operación de conversión

07 01 convert.png

Y pocos segundos después ya tenemos nuestro Tiny-YoloV3 en formato Onnx

07 convert running

Como el modelo respeta el Input / Output de la versión anterior, solo debemos reemplazar el archivo en nuestra solución. Yo he agregado el nuevo Onnx solo para tener un poco mas de control sobre el ejemplo.

08 Sol Onnx.png

Como siempre he actualizado el ejemplo completo en GitHub

https://github.com/elbruno/Blog/tree/master/20180709%20UwpMLNet%20TinyYoloV3

 

Happy Coding!

Saludos @ Burlington

El Bruno

References

#WinML – Problems With ONNX and Machine Learning.Net in the latest versions of #Windows10 Insiders

Hi!

After writing a step by step on how to use YoloV2 in a Windows App 10, I find that in the latest versions of Windows Insider, WinML It’s not working properly.

Windows 10 and YOLOV2 for Object Detection Series

 

I’ve reported the problem here Https://aka.ms/AA1sy9u And in my particular case in my Device With the version [17711.1000] as something fails in the UWPs App that use ONNX.

01

On another machine with version [17134.1] and it works properly

Happy Coding!

Greetings @ Toronto

El Bruno

References

#WinML – Problemas con ONNX y Machine Learning.Net en las versions de #Windows10 Insiders

Buenas!

Después de escribir un paso a paso sobre como utilizar YoloV2 en una App Windows 10, me encuentro con que en las ultimas versiones de Windows Insider, WinML no funciona correctamente.

Crear una Windows 10 UWP App y utilizar YoloV2 para reconocer objetos

He reportado el problema aquí https://aka.ms/AA1sy9u y en mi caso particular en mi device con la versión [17711.1000] pues algo falla en las UWPs app que utilizan ONNX.

01.png

En otra máquina con versión [17134.1] funciona correctamente

Happy Coding!

Saludos @ Toronto

El Bruno

References